[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:45:33 dist-test-slave-pww0 krb5kdc[25131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:45:33 dist-test-slave-pww0 krb5kdc[25131](info): set up 2 sockets
Aug 09 22:45:33 dist-test-slave-pww0 krb5kdc[25131](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 22:45:34.868160 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.006s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:45:34 dist-test-slave-pww0 krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:45:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:45:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:45:35.076311 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45805
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:35.424635 25149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:35.424937 25149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:35.425068 25149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:35.432611 25149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:45:35.432736 25149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:35.432814 25149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:35.432893 25149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:45:35.432955 25149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:45:35.443158 25149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:35.444593 25149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:35.446010 25149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:35.458743 25160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:35.458770 25157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:35.459018 25149 server_base.cc:1034] running on GCE node
W20260809 22:45:35.459386 25156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:35.460399 25149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:35.462090 25149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:35.463361 25149 hybrid_clock.cc:648] HybridClock initialized: now 1786315535463300 us; error 49 us; skew 500 ppm
I20260809 22:45:35.463779 25149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:35 dist-test-slave-pww0 krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315535, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:35.467434 25149 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:45:35.469120 25149 webserver.cc:460] Webserver started at http://127.24.136.62:45431/ using document root <none> and password file <none>
I20260809 22:45:35.469890 25149 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:35.470022 25149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:35.470372 25149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:35.473825 25149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "df8a2f324a08411f9a4db0cda1391cea"
format_stamp: "Formatted at 2026-08-09 22:45:35 on dist-test-slave-pww0"
server_key: "966b230ccbe41a927a318e772906a2a9"
server_key_iv: "bbea1c1ba09c2afbb474b3c4392ec89c"
server_key_version: "encryptionkey@0"
I20260809 22:45:35.474668 25149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "df8a2f324a08411f9a4db0cda1391cea"
format_stamp: "Formatted at 2026-08-09 22:45:35 on dist-test-slave-pww0"
server_key: "966b230ccbe41a927a318e772906a2a9"
server_key_iv: "bbea1c1ba09c2afbb474b3c4392ec89c"
server_key_version: "encryptionkey@0"
I20260809 22:45:35.480628 25149 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 22:45:35.484803 25172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:35.486409 25149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:45:35.486605 25149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "df8a2f324a08411f9a4db0cda1391cea"
format_stamp: "Formatted at 2026-08-09 22:45:35 on dist-test-slave-pww0"
server_key: "966b230ccbe41a927a318e772906a2a9"
server_key_iv: "bbea1c1ba09c2afbb474b3c4392ec89c"
server_key_version: "encryptionkey@0"
I20260809 22:45:35.486830 25149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:35.506440 25149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:35.511425 25149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:35.511775 25149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:35.530512 25149 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:45805
I20260809 22:45:35.530534 25267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:45805 every 8 connection(s)
I20260809 22:45:35.532099 25149 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:45:35.536350 25268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:35.537463 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 25149
I20260809 22:45:35.537792 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:45:35.539743 25120 external_mini_cluster.cc:1428] Setting key bc410926e1ce30b8501ba45d032c8883
I20260809 22:45:35.548255 25268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea: Bootstrap starting.
I20260809 22:45:35.551406 25268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:35.552498 25268 log.cc:824] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:35.555823 25268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea: No bootstrap required, opened a new log
I20260809 22:45:35.563709 25268 raft_consensus.cc:348] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8a2f324a08411f9a4db0cda1391cea" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45805 } }
I20260809 22:45:35.564131 25268 raft_consensus.cc:374] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:35.564234 25268 raft_consensus.cc:729] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df8a2f324a08411f9a4db0cda1391cea, State: Initialized, Role: FOLLOWER
I20260809 22:45:35.564909 25268 consensus_queue.cc:260] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8a2f324a08411f9a4db0cda1391cea" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45805 } }
I20260809 22:45:35.565205 25268 raft_consensus.cc:388] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:45:35.565354 25268 raft_consensus.cc:482] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:45:35.565519 25268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:35.567878 25268 raft_consensus.cc:504] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8a2f324a08411f9a4db0cda1391cea" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45805 } }
Aug 09 22:45:35 dist-test-slave-pww0 krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:35.568502 25268 leader_election.cc:302] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df8a2f324a08411f9a4db0cda1391cea; no voters:
I20260809 22:45:35.569151 25268 leader_election.cc:288] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:45:35.569284 25279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:35.570492 25279 raft_consensus.cc:686] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [term 1 LEADER]: Becoming Leader. State: Replica: df8a2f324a08411f9a4db0cda1391cea, State: Running, Role: LEADER
I20260809 22:45:35.571535 25279 consensus_queue.cc:237] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8a2f324a08411f9a4db0cda1391cea" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45805 } }
I20260809 22:45:35.571892 25268 sys_catalog.cc:564] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:45:35.575316 25281 sys_catalog.cc:455] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [sys.catalog]: SysCatalogTable state changed. Reason: New leader df8a2f324a08411f9a4db0cda1391cea. Latest consensus state: current_term: 1 leader_uuid: "df8a2f324a08411f9a4db0cda1391cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8a2f324a08411f9a4db0cda1391cea" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45805 } } }
I20260809 22:45:35.575915 25281 sys_catalog.cc:458] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:35.576202 25280 sys_catalog.cc:455] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df8a2f324a08411f9a4db0cda1391cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df8a2f324a08411f9a4db0cda1391cea" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45805 } } }
I20260809 22:45:35.576543 25280 sys_catalog.cc:458] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:35.581197 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:35.543807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39752 (local address 127.24.136.62:45805)
0809 22:45:35.544784 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:35.544807 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:35.545986 (+ 1179us) server_negotiation.cc:407] Connection header received
0809 22:45:35.555065 (+ 9079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:35.555088 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:35.555543 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:35.555949 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:35.559052 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:35.560642 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:35.562451 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:35.563176 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:35.563376 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:35.569721 (+ 6345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:35.569735 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:35.569762 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:35.569839 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:35.573871 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:35.575266 (+ 1395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:35.575278 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:35.575287 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:35.575575 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:35.576379 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:35.576384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:35.576388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:35.577495 (+ 1107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:35.577868 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:35.579121 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0809 22:45:35.579655 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":433,"threads_started":1}
I20260809 22:45:35.581936 25297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:45:35.589756 25297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:45:35.599817 25297 catalog_manager.cc:1347] Generated new cluster ID: c46c8ab4aa214ddb9e9cc160672e2659
I20260809 22:45:35.599973 25297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:45:35.611552 25297 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:45:35.613036 25297 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:45:35.624615 25297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df8a2f324a08411f9a4db0cda1391cea: Generated new TSK 0
I20260809 22:45:35.625595 25297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:45:35.689460 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45805
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:36.062927 25311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:36.063222 25311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:36.063331 25311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:36.070747 25311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:36.070890 25311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:36.071012 25311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:45:36.081439 25311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:36.083125 25311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:36.084534 25311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:36.098100 25328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:36.098119 25323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:36.099191 25311 server_base.cc:1034] running on GCE node
W20260809 22:45:36.099325 25322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:36.100062 25311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:36.101074 25311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:36.102277 25311 hybrid_clock.cc:648] HybridClock initialized: now 1786315536102226 us; error 37 us; skew 500 ppm
I20260809 22:45:36.102598 25311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:36 dist-test-slave-pww0 krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:36.106015 25311 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:45:36.107642 25311 webserver.cc:460] Webserver started at http://127.24.136.1:39255/ using document root <none> and password file <none>
I20260809 22:45:36.108385 25311 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:36.108494 25311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:36.108780 25311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:36.111912 25311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "7ab14e0559fe4e1d956f04b6ae7e0f3a"
format_stamp: "Formatted at 2026-08-09 22:45:36 on dist-test-slave-pww0"
server_key: "66ace8522879984bc82c1be2ddb4c2ae"
server_key_iv: "f5c24743c59f1a7dc9429fd9327acc12"
server_key_version: "encryptionkey@0"
I20260809 22:45:36.112666 25311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "7ab14e0559fe4e1d956f04b6ae7e0f3a"
format_stamp: "Formatted at 2026-08-09 22:45:36 on dist-test-slave-pww0"
server_key: "66ace8522879984bc82c1be2ddb4c2ae"
server_key_iv: "f5c24743c59f1a7dc9429fd9327acc12"
server_key_version: "encryptionkey@0"
I20260809 22:45:36.118265 25311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:45:36.122030 25343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:36.123725 25311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:45:36.123900 25311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "7ab14e0559fe4e1d956f04b6ae7e0f3a"
format_stamp: "Formatted at 2026-08-09 22:45:36 on dist-test-slave-pww0"
server_key: "66ace8522879984bc82c1be2ddb4c2ae"
server_key_iv: "f5c24743c59f1a7dc9429fd9327acc12"
server_key_version: "encryptionkey@0"
I20260809 22:45:36.124166 25311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:36.150900 25311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:36.156359 25311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:36.156651 25311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:36.157722 25311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:45:36.159230 25311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:36.159312 25311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:36.159411 25311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:36.159477 25311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:36.191864 25311 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:37467
I20260809 22:45:36.191927 25552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:37467 every 8 connection(s)
I20260809 22:45:36.193256 25311 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:45:36.193805 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 25311
I20260809 22:45:36.194032 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:45:36.194962 25120 external_mini_cluster.cc:1428] Setting key 4c86c2780253b261e20631c8f79ee884
Aug 09 22:45:36 dist-test-slave-pww0 krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
I20260809 22:45:36.215306 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:36.196407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:45233 (local address 127.24.136.62:45805)
0809 22:45:36.196902 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:36.196908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:36.198212 (+ 1304us) server_negotiation.cc:407] Connection header received
0809 22:45:36.200234 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:36.200239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:36.200342 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:36.200462 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:36.202249 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:36.203224 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:36.204971 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:36.205392 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:36.205510 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:36.209314 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:36.209323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:36.209327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:36.209378 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:36.212115 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:36.212964 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:36.212972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:36.212975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:36.213147 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:36.213611 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:36.213616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:36.213619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:36.213937 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:36.214066 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:36.214833 (+ 767us) server_negotiation.cc:299] Negotiation successful
0809 22:45:36.215023 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":266,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
I20260809 22:45:36.216536 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:36.197135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45805 (local address 127.24.136.1:45233)
0809 22:45:36.198005 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:36.198066 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:36.199935 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:36.200690 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:36.200727 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:36.201289 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:36.202042 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:36.202064 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:36.203410 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:36.203414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:36.204814 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:36.204828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:36.205653 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:36.205661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:36.205889 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:36.206561 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:36.206587 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:36.209029 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:36.212278 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:36.212289 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:36.212310 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:36.212786 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:36.213267 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:36.213272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:36.213275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:36.213490 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:36.214064 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:36.214070 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:36.214434 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 22:45:36.214710 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 22:45:36.215215 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":340,"threads_started":1}
I20260809 22:45:36.218361 25554 heartbeater.cc:347] Connected to a master server at 127.24.136.62:45805
I20260809 22:45:36.218681 25554 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:36.219480 25554 heartbeater.cc:511] Master 127.24.136.62:45805 requested a full tablet report, sending...
I20260809 22:45:36.222039 25204 ts_manager.cc:194] Registered new tserver with Master: 7ab14e0559fe4e1d956f04b6ae7e0f3a (127.24.136.1:37467)
I20260809 22:45:36.224185 25204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:45233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:45:36.242518 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45805
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:36.597982 25565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:36.598326 25565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:36.598476 25565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:36.606418 25565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:36.606562 25565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:36.606727 25565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:45:36.617197 25565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:36.618592 25565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:36.620019 25565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:36.633517 25571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:36.633558 25574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:36.633536 25576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:36.634320 25565 server_base.cc:1034] running on GCE node
I20260809 22:45:36.634964 25565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:36.636029 25565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:36.637252 25565 hybrid_clock.cc:648] HybridClock initialized: now 1786315536637182 us; error 59 us; skew 500 ppm
I20260809 22:45:36.637607 25565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:36 dist-test-slave-pww0 krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:36.640898 25565 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:45:36.642606 25565 webserver.cc:460] Webserver started at http://127.24.136.2:45125/ using document root <none> and password file <none>
I20260809 22:45:36.643397 25565 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:36.643565 25565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:36.643901 25565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:36.647101 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "6f99c1b76a8d43a18ee57f76400000ad"
format_stamp: "Formatted at 2026-08-09 22:45:36 on dist-test-slave-pww0"
server_key: "d9f2d2e9db0b7d2d51c2f2ec1bbe660c"
server_key_iv: "76316aa89b4c13e125343b71804d8465"
server_key_version: "encryptionkey@0"
I20260809 22:45:36.647995 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "6f99c1b76a8d43a18ee57f76400000ad"
format_stamp: "Formatted at 2026-08-09 22:45:36 on dist-test-slave-pww0"
server_key: "d9f2d2e9db0b7d2d51c2f2ec1bbe660c"
server_key_iv: "76316aa89b4c13e125343b71804d8465"
server_key_version: "encryptionkey@0"
I20260809 22:45:36.653793 25565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:45:36.657915 25587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:36.659384 25565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:45:36.659644 25565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "6f99c1b76a8d43a18ee57f76400000ad"
format_stamp: "Formatted at 2026-08-09 22:45:36 on dist-test-slave-pww0"
server_key: "d9f2d2e9db0b7d2d51c2f2ec1bbe660c"
server_key_iv: "76316aa89b4c13e125343b71804d8465"
server_key_version: "encryptionkey@0"
I20260809 22:45:36.659855 25565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:36.692250 25565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:36.697029 25565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:36.697348 25565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:36.698498 25565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:45:36.700039 25565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:36.700129 25565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:36.700269 25565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:36.700326 25565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:36.755337 25565 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:34133
I20260809 22:45:36.755425 25796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:34133 every 8 connection(s)
I20260809 22:45:36.756798 25565 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:45:36.764688 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 25565
I20260809 22:45:36.764925 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:45:36.765496 25120 external_mini_cluster.cc:1428] Setting key f3d8f8c3f12157077be8d8c631944c26
Aug 09 22:45:36 dist-test-slave-pww0 krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:36.783159 25808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:36.760027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:51763 (local address 127.24.136.62:45805)
0809 22:45:36.760584 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:36.760591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:36.761960 (+ 1369us) server_negotiation.cc:407] Connection header received
0809 22:45:36.764974 (+ 3014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:36.764981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:36.765086 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:36.765221 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:36.767680 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:36.768582 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:36.771273 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:36.772064 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:36.772163 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:36.776357 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:36.776365 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:36.776370 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:36.776435 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:36.779736 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:36.780572 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:36.780579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:36.780583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:36.780805 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:36.781243 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:36.781247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:36.781250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:36.781594 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:36.781742 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:36.782503 (+ 761us) server_negotiation.cc:299] Negotiation successful
0809 22:45:36.782760 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":271,"threads_started":1}
I20260809 22:45:36.784073 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:36.760687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45805 (local address 127.24.136.2:51763)
0809 22:45:36.761717 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:36.761790 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:36.764553 (+ 2763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:36.765544 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:36.765559 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:36.766442 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:36.767429 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:36.767493 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:36.768783 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:36.768789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:36.771047 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:36.771067 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:36.772560 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:36.772567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:36.772809 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:36.773555 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:36.773588 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:36.776078 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:36.779902 (+ 3824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:36.779912 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:36.779928 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:36.780409 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:36.780916 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:36.780919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:36.780921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:36.781121 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:36.781701 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:36.781707 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:36.782091 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 22:45:36.782343 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 22:45:36.782793 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":381,"threads_started":1}
I20260809 22:45:36.785691 25800 heartbeater.cc:347] Connected to a master server at 127.24.136.62:45805
I20260809 22:45:36.786018 25800 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:36.786793 25800 heartbeater.cc:511] Master 127.24.136.62:45805 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
I20260809 22:45:36.788443 25204 ts_manager.cc:194] Registered new tserver with Master: 6f99c1b76a8d43a18ee57f76400000ad (127.24.136.2:34133)
I20260809 22:45:36.789300 25204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:51763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:45:36.814226 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45805
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:37.169456 25817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:37.169822 25817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:37.170091 25817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:37.178687 25817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:37.178853 25817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:37.179013 25817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:45:37.189440 25817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:37.190918 25817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:37.192442 25817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:37.206403 25827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:37.206099 25828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:37.206041 25832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:37.207484 25817 server_base.cc:1034] running on GCE node
I20260809 22:45:37.208194 25817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:37.209265 25817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:37.210512 25817 hybrid_clock.cc:648] HybridClock initialized: now 1786315537210440 us; error 62 us; skew 500 ppm
I20260809 22:45:37.210929 25817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:37 dist-test-slave-pww0 krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315537, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:37.214771 25817 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:45:37.216528 25817 webserver.cc:460] Webserver started at http://127.24.136.3:35777/ using document root <none> and password file <none>
I20260809 22:45:37.217315 25817 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:37.217451 25817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:37.217797 25817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:37.221206 25817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "ec5a35f805fa40d3ae964e4614e45471"
format_stamp: "Formatted at 2026-08-09 22:45:37 on dist-test-slave-pww0"
server_key: "8b96eed0a45a09fc59a10340bf8ff9cb"
server_key_iv: "deba944cfe6b8055e7f504e291590f26"
server_key_version: "encryptionkey@0"
I20260809 22:45:37.222066 25817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "ec5a35f805fa40d3ae964e4614e45471"
format_stamp: "Formatted at 2026-08-09 22:45:37 on dist-test-slave-pww0"
server_key: "8b96eed0a45a09fc59a10340bf8ff9cb"
server_key_iv: "deba944cfe6b8055e7f504e291590f26"
server_key_version: "encryptionkey@0"
I20260809 22:45:37.228024 25817 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 22:45:37.228107 25554 heartbeater.cc:503] Master 127.24.136.62:45805 was elected leader, sending a full tablet report...
I20260809 22:45:37.232501 25845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:37.234017 25817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:45:37.234182 25817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "ec5a35f805fa40d3ae964e4614e45471"
format_stamp: "Formatted at 2026-08-09 22:45:37 on dist-test-slave-pww0"
server_key: "8b96eed0a45a09fc59a10340bf8ff9cb"
server_key_iv: "deba944cfe6b8055e7f504e291590f26"
server_key_version: "encryptionkey@0"
I20260809 22:45:37.234395 25817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:37.249287 25817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:37.253979 25817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:37.254311 25817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:37.255431 25817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:45:37.257076 25817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:37.257159 25817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:37.257287 25817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:37.257345 25817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:37.290493 25817 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:35417
I20260809 22:45:37.290529 26024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:35417 every 8 connection(s)
I20260809 22:45:37.292140 25817 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:45:37.295248 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 25817
I20260809 22:45:37.295426 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:45:37.295946 25120 external_mini_cluster.cc:1428] Setting key a1bcc4fa8e7023d6738b296a95a5d3e1
Aug 09 22:45:37 dist-test-slave-pww0 krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315537, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:37.313982 26030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:37.294766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:60729 (local address 127.24.136.62:45805)
0809 22:45:37.295278 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:37.295284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:37.296503 (+ 1219us) server_negotiation.cc:407] Connection header received
0809 22:45:37.298873 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:37.298879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:37.298972 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:37.299097 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:37.300893 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.301701 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.303301 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.303754 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.303845 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:37.307373 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:37.307378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:37.307381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:37.307426 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:37.310122 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.311186 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.311192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.311195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.311355 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.311848 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.311854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.311857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.312254 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:37.312432 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:37.313471 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0809 22:45:37.313674 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":241,"threads_started":1}
I20260809 22:45:37.314956 26031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:37.295389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45805 (local address 127.24.136.3:60729)
0809 22:45:37.296309 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:37.296366 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:37.298539 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:37.299287 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:37.299296 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:37.299972 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:37.300698 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:37.300724 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.301888 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.301892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:37.303181 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:37.303194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.303956 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.303961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:37.304161 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:37.304794 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:37.304817 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:37.307133 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:37.310351 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:37.310364 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:37.310389 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:37.311076 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:37.311497 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:37.311503 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:37.311507 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:37.311734 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:37.312424 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:37.312432 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:37.312957 (+ 525us) client_negotiation.cc:769] Sending connection context
0809 22:45:37.313325 (+ 368us) client_negotiation.cc:240] Negotiation successful
0809 22:45:37.313782 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":315,"threads_started":1}
I20260809 22:45:37.316478 26025 heartbeater.cc:347] Connected to a master server at 127.24.136.62:45805
I20260809 22:45:37.316829 26025 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:37.317600 26025 heartbeater.cc:511] Master 127.24.136.62:45805 requested a full tablet report, sending...
I20260809 22:45:37.319272 25204 ts_manager.cc:194] Registered new tserver with Master: ec5a35f805fa40d3ae964e4614e45471 (127.24.136.3:35417)
I20260809 22:45:37.320163 25204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:60729
I20260809 22:45:37.326038 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:45:37 dist-test-slave-pww0 krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315537, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:45:37 dist-test-slave-pww0 krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315537, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:45:37.379035 26040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:37.362577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42500 (local address 127.24.136.1:37467)
0809 22:45:37.363384 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:37.363395 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:37.363425 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:45:37.363532 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:37.363540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:37.363775 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:37.363960 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:37.365538 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.366581 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.368012 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.368614 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.368739 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:37.372870 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:37.372885 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:37.372901 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:37.372948 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:37.375904 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.376710 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.376719 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.376722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.376877 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.377316 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.377320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.377322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.377704 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:37.378028 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:37.378371 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 22:45:37.378661 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":379,"threads_started":1}
W20260809 22:45:37.380421 25396 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42500
Aug 09 22:45:37 dist-test-slave-pww0 krb5kdc[25131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:45:37 dist-test-slave-pww0 krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:45:37.431005 26040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:37.415768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.24.136.1:37467)
0809 22:45:37.416005 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:37.416011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:37.416176 (+ 165us) server_negotiation.cc:407] Connection header received
0809 22:45:37.416375 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:37.416379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:37.416478 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:37.416568 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:37.418025 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.418801 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.420138 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.420653 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.420798 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:37.425505 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:37.425511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:37.425514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:37.425555 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:37.428306 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.429004 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.429009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.429011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.429201 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.429584 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.429588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.429590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.429874 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:37.430105 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:37.430450 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 22:45:37.430694 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:45:37.432801 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 25311
I20260809 22:45:37.446169 26040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:37.433766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.24.136.1:37467)
0809 22:45:37.433942 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:37.433947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:37.434288 (+ 341us) server_negotiation.cc:407] Connection header received
0809 22:45:37.434414 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:37.434417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:37.434479 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:37.434559 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:37.435861 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.436631 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.438041 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.438531 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.438646 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:37.441046 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:37.441051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:37.441055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:37.441093 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:37.443723 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.444432 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.444437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.444440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.444587 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.444889 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.444891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.444893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.445159 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:37.445356 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:37.445705 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 22:45:37.445875 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:45:37.446898 25545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:37.530835 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 25311
I20260809 22:45:37.555322 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 25565
Aug 09 22:45:37 dist-test-slave-pww0 krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:45:37.572791 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:37.556336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58546 (local address 127.24.136.2:34133)
0809 22:45:37.556902 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:37.556909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:37.556936 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:45:37.557005 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:37.557012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:37.557218 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:37.557392 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:37.558631 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.559719 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.561011 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.561592 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.561712 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:37.566164 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:37.566180 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:37.566198 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:37.566241 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:37.569467 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.570398 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.570407 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.570410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.570574 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.570969 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.570973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.570975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.571416 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:37.571787 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:37.572198 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 22:45:37.572448 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":279,"threads_started":1}
I20260809 22:45:37.574232 25788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:37.657418 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 25565
I20260809 22:45:37.679934 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 25817
Aug 09 22:45:37 dist-test-slave-pww0 krb5kdc[25131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:45:37.697969 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:37.681005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.24.136.3:35417)
0809 22:45:37.681728 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:37.681736 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:37.681768 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:45:37.681836 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:37.681843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:37.682067 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:37.682210 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:37.683617 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.684757 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.686203 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:37.686829 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:37.686944 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:37.691681 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:37.691700 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:37.691718 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:37.691770 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:37.694838 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.695652 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.695682 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.695685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.695828 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:37.696279 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:37.696283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:37.696284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:37.696692 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:37.696956 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:37.697312 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 22:45:37.697586 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":356,"threads_started":1}
I20260809 22:45:37.699349 26020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:37.782466 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 25817
I20260809 22:45:37.803241 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 25149
I20260809 22:45:37.804522 25261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:37.909294 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 25149
2026-08-09T22:45:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4371 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:45:38 dist-test-slave-pww0 krb5kdc[26063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:45:38 dist-test-slave-pww0 krb5kdc[26063](info): set up 2 sockets
Aug 09 22:45:38 dist-test-slave-pww0 krb5kdc[26063](info): commencing operation
krb5kdc: starting...
W20260809 22:45:39.161159 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:45:39 dist-test-slave-pww0 krb5kdc[26063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:45:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:45:39.371228 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45543
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:39.707424 26079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:39.707769 26079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:39.707907 26079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:39.715411 26079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:45:39.715583 26079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:39.715669 26079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:39.715775 26079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:45:39.715842 26079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:45:39.726233 26079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:39.727713 26079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:39.729125 26079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:39.741823 26092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:39.742031 26095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:39.742460 26098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:39.743016 26079 server_base.cc:1034] running on GCE node
I20260809 22:45:39.744086 26079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:39.745726 26079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:39.746960 26079 hybrid_clock.cc:648] HybridClock initialized: now 1786315539746901 us; error 55 us; skew 500 ppm
I20260809 22:45:39.747345 26079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:39 dist-test-slave-pww0 krb5kdc[26063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315539, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:39.756094 26079 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:45:39.758075 26079 webserver.cc:460] Webserver started at http://127.24.136.62:39885/ using document root <none> and password file <none>
I20260809 22:45:39.758857 26079 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:39.758985 26079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:39.759341 26079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:39.762683 26079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "018eacaa0b884a829c88e100fa2bc1bb"
format_stamp: "Formatted at 2026-08-09 22:45:39 on dist-test-slave-pww0"
server_key: "f069ff7a42b4ed59994ec4656510d91b"
server_key_iv: "03e01e64ece95faaeac87cc3dd525ae5"
server_key_version: "encryptionkey@0"
I20260809 22:45:39.763538 26079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "018eacaa0b884a829c88e100fa2bc1bb"
format_stamp: "Formatted at 2026-08-09 22:45:39 on dist-test-slave-pww0"
server_key: "f069ff7a42b4ed59994ec4656510d91b"
server_key_iv: "03e01e64ece95faaeac87cc3dd525ae5"
server_key_version: "encryptionkey@0"
I20260809 22:45:39.769402 26079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:45:39.773460 26110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:39.774886 26079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:45:39.775063 26079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "018eacaa0b884a829c88e100fa2bc1bb"
format_stamp: "Formatted at 2026-08-09 22:45:39 on dist-test-slave-pww0"
server_key: "f069ff7a42b4ed59994ec4656510d91b"
server_key_iv: "03e01e64ece95faaeac87cc3dd525ae5"
server_key_version: "encryptionkey@0"
I20260809 22:45:39.775280 26079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:39.804128 26079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:39.808660 26079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:39.808969 26079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:39.827684 26079 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:45543
I20260809 22:45:39.827693 26210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:45543 every 8 connection(s)
I20260809 22:45:39.829233 26079 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:45:39.832593 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 26079
I20260809 22:45:39.832821 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:45:39.833483 25120 external_mini_cluster.cc:1428] Setting key da43d550689ec773b364ee4f4f3af331
I20260809 22:45:39.833523 26211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:39.848595 26211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb: Bootstrap starting.
Aug 09 22:45:39 dist-test-slave-pww0 krb5kdc[26063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:39.852586 26211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:39.853852 26211 log.cc:824] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:39.857645 26211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb: No bootstrap required, opened a new log
I20260809 22:45:39.858480 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:39.835642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56912 (local address 127.24.136.62:45543)
0809 22:45:39.836530 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:39.836545 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:39.836600 (+ 55us) server_negotiation.cc:407] Connection header received
0809 22:45:39.838573 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:39.838605 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:39.839175 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:39.839665 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:39.841014 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:39.843014 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:39.844393 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:39.845538 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:39.845770 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:39.850058 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:39.850067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:39.850083 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:39.850127 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:39.853507 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:39.854500 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:39.854511 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:39.854523 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:39.854855 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:39.855254 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:39.855264 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:39.855268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:39.856001 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:39.856246 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:39.856883 (+ 637us) server_negotiation.cc:299] Negotiation successful
0809 22:45:39.857309 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":327,"threads_started":1}
I20260809 22:45:39.866261 26211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018eacaa0b884a829c88e100fa2bc1bb" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45543 } }
I20260809 22:45:39.866763 26211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:39.866871 26211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018eacaa0b884a829c88e100fa2bc1bb, State: Initialized, Role: FOLLOWER
I20260809 22:45:39.867558 26211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018eacaa0b884a829c88e100fa2bc1bb" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45543 } }
I20260809 22:45:39.867825 26211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:45:39.867961 26211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:45:39.868111 26211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:39.870417 26211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018eacaa0b884a829c88e100fa2bc1bb" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45543 } }
I20260809 22:45:39.870988 26211 leader_election.cc:302] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 018eacaa0b884a829c88e100fa2bc1bb; no voters:
I20260809 22:45:39.871673 26211 leader_election.cc:288] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:45:39.871871 26219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:39.872551 26219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [term 1 LEADER]: Becoming Leader. State: Replica: 018eacaa0b884a829c88e100fa2bc1bb, State: Running, Role: LEADER
I20260809 22:45:39.873113 26219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018eacaa0b884a829c88e100fa2bc1bb" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45543 } }
I20260809 22:45:39.874253 26211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:45:39.876638 26224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 018eacaa0b884a829c88e100fa2bc1bb. Latest consensus state: current_term: 1 leader_uuid: "018eacaa0b884a829c88e100fa2bc1bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018eacaa0b884a829c88e100fa2bc1bb" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45543 } } }
I20260809 22:45:39.877156 26224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:39.877728 26222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "018eacaa0b884a829c88e100fa2bc1bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "018eacaa0b884a829c88e100fa2bc1bb" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45543 } } }
I20260809 22:45:39.878031 26222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:39.879951 26236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:45:39.886003 26236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:45:39.896476 26236 catalog_manager.cc:1347] Generated new cluster ID: d25f52576d0445c0b25ca0b5de9105a3
I20260809 22:45:39.896613 26236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:45:39.925145 26236 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:45:39.927172 26236 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:45:39.943711 26236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 018eacaa0b884a829c88e100fa2bc1bb: Generated new TSK 0
I20260809 22:45:39.944967 26236 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:45:40.003448 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45543
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:40.364364 26256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:40.364709 26256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:40.364877 26256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:40.372609 26256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:40.372748 26256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:40.372870 26256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:45:40.383944 26256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45543
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:40.385393 26256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:40.386795 26256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:40.401242 26265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:40.401181 26266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:40.401188 26268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:40.402194 26256 server_base.cc:1034] running on GCE node
I20260809 22:45:40.402827 26256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:40.403906 26256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:40.405153 26256 hybrid_clock.cc:648] HybridClock initialized: now 1786315540405084 us; error 52 us; skew 500 ppm
I20260809 22:45:40.405524 26256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:40 dist-test-slave-pww0 krb5kdc[26063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:40.409060 26256 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:45:40.411100 26256 webserver.cc:460] Webserver started at http://127.24.136.1:34875/ using document root <none> and password file <none>
I20260809 22:45:40.411919 26256 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:40.412034 26256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:40.412320 26256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:40.415519 26256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "abb7c319dee14ff3b893e6666dff81ca"
format_stamp: "Formatted at 2026-08-09 22:45:40 on dist-test-slave-pww0"
server_key: "6e099c65f5c9b7d87fde7ad1a30f2f0b"
server_key_iv: "0f5f84816b432c67ef03ab8cbd739375"
server_key_version: "encryptionkey@0"
I20260809 22:45:40.416355 26256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "abb7c319dee14ff3b893e6666dff81ca"
format_stamp: "Formatted at 2026-08-09 22:45:40 on dist-test-slave-pww0"
server_key: "6e099c65f5c9b7d87fde7ad1a30f2f0b"
server_key_iv: "0f5f84816b432c67ef03ab8cbd739375"
server_key_version: "encryptionkey@0"
I20260809 22:45:40.422106 26256 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 22:45:40.426276 26286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:40.427923 26256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:45:40.428121 26256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "abb7c319dee14ff3b893e6666dff81ca"
format_stamp: "Formatted at 2026-08-09 22:45:40 on dist-test-slave-pww0"
server_key: "6e099c65f5c9b7d87fde7ad1a30f2f0b"
server_key_iv: "0f5f84816b432c67ef03ab8cbd739375"
server_key_version: "encryptionkey@0"
I20260809 22:45:40.428335 26256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:40.442112 26256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:40.446936 26256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:40.447245 26256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:40.448329 26256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:45:40.449849 26256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:40.449941 26256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:40.450071 26256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:40.450129 26256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:40.483304 26256 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:37485
I20260809 22:45:40.483355 26483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:37485 every 8 connection(s)
I20260809 22:45:40.484743 26256 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:45:40.487818 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 26256
I20260809 22:45:40.488041 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:45:40.488752 25120 external_mini_cluster.cc:1428] Setting key 4423b64fdfe39df255f450fb89250521
Aug 09 22:45:40 dist-test-slave-pww0 krb5kdc[26063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
I20260809 22:45:40.508350 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:40.487686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:33813 (local address 127.24.136.62:45543)
0809 22:45:40.488260 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:40.488267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:40.489310 (+ 1043us) server_negotiation.cc:407] Connection header received
0809 22:45:40.491627 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:40.491633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:40.491725 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:40.492012 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:40.493909 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:40.494710 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:40.496602 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:40.497049 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:40.497181 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:40.501845 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:40.501852 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:40.501855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:40.501898 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:40.504676 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:40.505652 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:40.505657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:40.505660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:40.505813 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:40.506361 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:40.506366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:40.506368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:40.506674 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:40.506840 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:40.507710 (+ 870us) server_negotiation.cc:299] Negotiation successful
0809 22:45:40.507975 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":268,"threads_started":1}
I20260809 22:45:40.509413 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:40.488231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45543 (local address 127.24.136.1:33813)
0809 22:45:40.489108 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:40.489159 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:40.491277 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:40.492308 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:40.492319 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:40.492967 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:40.493645 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:40.493676 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:40.494910 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:40.494915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:40.496391 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:40.496414 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:40.497339 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:40.497347 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:40.497617 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:40.498725 (+ 1108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:40.498753 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:40.501515 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:40.504912 (+ 3397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:40.504922 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:40.504939 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:40.505479 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:40.505951 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:40.505955 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:40.505957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:40.506210 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:40.506842 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:40.506852 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:40.507252 (+ 400us) client_negotiation.cc:769] Sending connection context
0809 22:45:40.507557 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 22:45:40.508170 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":366,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
I20260809 22:45:40.511700 26484 heartbeater.cc:347] Connected to a master server at 127.24.136.62:45543
I20260809 22:45:40.512132 26484 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:40.513144 26484 heartbeater.cc:511] Master 127.24.136.62:45543 requested a full tablet report, sending...
I20260809 22:45:40.515717 26146 ts_manager.cc:194] Registered new tserver with Master: abb7c319dee14ff3b893e6666dff81ca (127.24.136.1:37485)
I20260809 22:45:40.517256 26146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:33813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:45:40.538822 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45543
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:40.892825 26498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:40.893148 26498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:40.893251 26498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:40.900802 26498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:40.900938 26498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:40.901059 26498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:45:40.911419 26498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45543
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:40.912810 26498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:40.914322 26498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:40.928814 26507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:40.928854 26509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:40.929001 26511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:40.929600 26498 server_base.cc:1034] running on GCE node
I20260809 22:45:40.930346 26498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:40.931592 26498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:40.932821 26498 hybrid_clock.cc:648] HybridClock initialized: now 1786315540932761 us; error 54 us; skew 500 ppm
I20260809 22:45:40.933176 26498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:40 dist-test-slave-pww0 krb5kdc[26063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:40.936568 26498 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:45:40.938166 26498 webserver.cc:460] Webserver started at http://127.24.136.2:36313/ using document root <none> and password file <none>
I20260809 22:45:40.938931 26498 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:40.939064 26498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:40.939384 26498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:40.942796 26498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "db53f482487d4442a8dbc0ec8c495356"
format_stamp: "Formatted at 2026-08-09 22:45:40 on dist-test-slave-pww0"
server_key: "c9c00c8be7975dd5ea6b63d6836f9fe9"
server_key_iv: "53497edd34ec4ec4de9f2e35fbc19767"
server_key_version: "encryptionkey@0"
I20260809 22:45:40.943665 26498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "db53f482487d4442a8dbc0ec8c495356"
format_stamp: "Formatted at 2026-08-09 22:45:40 on dist-test-slave-pww0"
server_key: "c9c00c8be7975dd5ea6b63d6836f9fe9"
server_key_iv: "53497edd34ec4ec4de9f2e35fbc19767"
server_key_version: "encryptionkey@0"
I20260809 22:45:40.949219 26498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:45:40.953120 26522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:40.954699 26498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:45:40.954898 26498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "db53f482487d4442a8dbc0ec8c495356"
format_stamp: "Formatted at 2026-08-09 22:45:40 on dist-test-slave-pww0"
server_key: "c9c00c8be7975dd5ea6b63d6836f9fe9"
server_key_iv: "53497edd34ec4ec4de9f2e35fbc19767"
server_key_version: "encryptionkey@0"
I20260809 22:45:40.955111 26498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:40.973304 26498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:40.978109 26498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:40.978410 26498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:40.979579 26498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:45:40.981113 26498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:40.981200 26498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:40.981346 26498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:40.981402 26498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:41.013703 26498 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:38577
I20260809 22:45:41.013773 26733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:38577 every 8 connection(s)
I20260809 22:45:41.015091 26498 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:45:41.021943 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 26498
I20260809 22:45:41.022157 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:45:41.022670 25120 external_mini_cluster.cc:1428] Setting key e3ea26a1cdbd77ffc04149fca945b5c3
Aug 09 22:45:41 dist-test-slave-pww0 krb5kdc[26063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315540, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:41.037456 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.017923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:48125 (local address 127.24.136.62:45543)
0809 22:45:41.018368 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.018373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.019438 (+ 1065us) server_negotiation.cc:407] Connection header received
0809 22:45:41.021561 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.021567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.021663 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.021766 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:41.023902 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.024674 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.026360 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.026816 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.026931 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.030725 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:41.030731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:41.030735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:41.030772 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:41.033638 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.034706 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.034712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.034714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.034856 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.035372 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.035375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.035377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.035746 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:41.035884 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.036974 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.037187 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":217,"threads_started":1}
I20260809 22:45:41.038967 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.018493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45543 (local address 127.24.136.2:48125)
0809 22:45:41.019259 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:41.019306 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:41.021224 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:41.022082 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:41.022094 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:41.022868 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:41.023699 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.023726 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.024860 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.024864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.026188 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.026203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.027077 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.027086 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.027329 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.028064 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:41.028089 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:41.030439 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:41.033821 (+ 3382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:41.033831 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:41.033853 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:41.034536 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:41.034978 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:41.034982 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:41.034986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:41.035255 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:41.035882 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:41.035890 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:41.036455 (+ 565us) client_negotiation.cc:769] Sending connection context
0809 22:45:41.036862 (+ 407us) client_negotiation.cc:240] Negotiation successful
0809 22:45:41.037464 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":306,"threads_started":1}
I20260809 22:45:41.040661 26734 heartbeater.cc:347] Connected to a master server at 127.24.136.62:45543
I20260809 22:45:41.040971 26734 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:41.041755 26734 heartbeater.cc:511] Master 127.24.136.62:45543 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
I20260809 22:45:41.043427 26146 ts_manager.cc:194] Registered new tserver with Master: db53f482487d4442a8dbc0ec8c495356 (127.24.136.2:38577)
I20260809 22:45:41.044302 26146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:48125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:45:41.071871 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45543
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:41.420333 26746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:41.420626 26746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:41.420763 26746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:41.428404 26746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:41.428535 26746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:41.428694 26746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:45:41.440086 26746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45543
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:41.441464 26746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:41.442884 26746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:41.456578 26756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:41.456643 26754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:41.456720 26759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:41.457809 26746 server_base.cc:1034] running on GCE node
I20260809 22:45:41.458508 26746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:41.459591 26746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:41.460810 26746 hybrid_clock.cc:648] HybridClock initialized: now 1786315541460749 us; error 50 us; skew 500 ppm
I20260809 22:45:41.461135 26746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:41 dist-test-slave-pww0 krb5kdc[26063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:41.464349 26746 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:45:41.465904 26746 webserver.cc:460] Webserver started at http://127.24.136.3:34835/ using document root <none> and password file <none>
I20260809 22:45:41.466650 26746 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:41.466763 26746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:41.467108 26746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:41.470383 26746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "ca8ece2e19534a8895610e4ac254a4da"
format_stamp: "Formatted at 2026-08-09 22:45:41 on dist-test-slave-pww0"
server_key: "15689ad7bf990f236420a10aa4d6907e"
server_key_iv: "022d8795184b8f8e9d8b8a7726d61baa"
server_key_version: "encryptionkey@0"
I20260809 22:45:41.471248 26746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "ca8ece2e19534a8895610e4ac254a4da"
format_stamp: "Formatted at 2026-08-09 22:45:41 on dist-test-slave-pww0"
server_key: "15689ad7bf990f236420a10aa4d6907e"
server_key_iv: "022d8795184b8f8e9d8b8a7726d61baa"
server_key_version: "encryptionkey@0"
I20260809 22:45:41.476858 26746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:45:41.480738 26767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:41.482223 26746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:45:41.482417 26746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "ca8ece2e19534a8895610e4ac254a4da"
format_stamp: "Formatted at 2026-08-09 22:45:41 on dist-test-slave-pww0"
server_key: "15689ad7bf990f236420a10aa4d6907e"
server_key_iv: "022d8795184b8f8e9d8b8a7726d61baa"
server_key_version: "encryptionkey@0"
I20260809 22:45:41.482671 26746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:41.509945 26746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:41.514595 26746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:41.514941 26746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:41.516064 26746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:45:41.517625 26746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:41.517707 26746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:41.517836 26746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:41.517894 26746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:41.520692 26484 heartbeater.cc:503] Master 127.24.136.62:45543 was elected leader, sending a full tablet report...
I20260809 22:45:41.550954 26746 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:38527
I20260809 22:45:41.551007 26958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:38527 every 8 connection(s)
I20260809 22:45:41.552352 26746 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:45:41.554186 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 26746
I20260809 22:45:41.554390 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:45:41.554943 25120 external_mini_cluster.cc:1428] Setting key 3f42b0fd95b325094e0a8b208efcba54
Aug 09 22:45:41 dist-test-slave-pww0 krb5kdc[26063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:41.573879 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.555527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53731 (local address 127.24.136.62:45543)
0809 22:45:41.556059 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.556064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.557180 (+ 1116us) server_negotiation.cc:407] Connection header received
0809 22:45:41.559310 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.559315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.559386 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.559531 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:41.561096 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.561950 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.563610 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.564030 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.564149 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.567632 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:41.567638 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:41.567642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:41.567676 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:41.570504 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.571399 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.571405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.571407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.571593 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.572133 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.572137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.572140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.572436 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:41.572563 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.573368 (+ 805us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.573580 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":264,"threads_started":1}
I20260809 22:45:41.574966 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.556140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45543 (local address 127.24.136.3:53731)
0809 22:45:41.557010 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:41.557054 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:41.559041 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:41.559758 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:41.559767 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:41.560310 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:41.560902 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.560919 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.562091 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.562095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.563422 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.563435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.564264 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.564270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.564473 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.565143 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:41.565167 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:41.567386 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:41.570679 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:41.570690 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:41.570730 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:41.571243 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:41.571758 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:41.571763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:41.571765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:41.571999 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:41.572588 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:41.572596 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:41.572968 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 22:45:41.573221 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 22:45:41.573731 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":322,"threads_started":1}
I20260809 22:45:41.576505 26960 heartbeater.cc:347] Connected to a master server at 127.24.136.62:45543
I20260809 22:45:41.576808 26960 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:41.577579 26960 heartbeater.cc:511] Master 127.24.136.62:45543 requested a full tablet report, sending...
I20260809 22:45:41.579351 26146 ts_manager.cc:194] Registered new tserver with Master: ca8ece2e19534a8895610e4ac254a4da (127.24.136.3:38527)
I20260809 22:45:41.580263 26146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:53731
I20260809 22:45:41.583614 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:45:41.602919 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.590010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.24.136.62:45543)
0809 22:45:41.590258 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.590265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.590595 (+ 330us) server_negotiation.cc:407] Connection header received
0809 22:45:41.590744 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.590748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.590829 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.590914 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:41.592553 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.593302 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.594629 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.595061 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.595157 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.597565 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:41.597571 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:41.597574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:41.597621 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:41.600139 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.600846 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.600850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.600853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.600981 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.601386 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.601390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.601393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.601743 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:41.601969 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.602363 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.602606 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:45:41.609651 26146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:45:41.616825 26146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:41.641073 26975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.630178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:37485 (local address 127.0.0.1:43388)
0809 22:45:41.631160 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:41.631225 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:41.631418 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:41.633297 (+ 1879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:41.633302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:41.633329 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:41.633840 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.633854 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.635994 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.635999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.638240 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.638251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.640038 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.640045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.640264 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.640302 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:45:41.640378 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 22:45:41.640645 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":16,"thread_start_us":541,"threads_started":1}
I20260809 22:45:41.641167 26980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.631022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51860 (local address 127.24.136.2:38577)
0809 22:45:41.632272 (+ 1250us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.632282 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.632315 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:45:41.632420 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.632429 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.632708 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.632909 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:41.634095 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.635489 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.638401 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.639638 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.639774 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.640558 (+ 784us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.640647 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.640834 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1081,"thread_start_us":1806,"threads_started":1}
I20260809 22:45:41.641834 26981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.632097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:38527 (local address 127.0.0.1:35908)
0809 22:45:41.632977 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:41.632993 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:41.633094 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:41.634275 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:41.634280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:41.634304 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:41.634672 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.634683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.636712 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.636717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.638746 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.638762 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.640632 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.640638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.640803 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.640836 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:45:41.640901 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 22:45:41.641028 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":63,"thread_start_us":890,"threads_started":1}
I20260809 22:45:41.641986 26984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.632678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.24.136.3:38527)
0809 22:45:41.633646 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.633656 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.633754 (+ 98us) server_negotiation.cc:407] Connection header received
0809 22:45:41.633851 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.633858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.634066 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.634887 (+ 821us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:41.635149 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.636539 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.638942 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.640399 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.641184 (+ 785us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.641438 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.641507 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.641678 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":635,"threads_started":1}
I20260809 22:45:41.642428 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.630539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.24.136.1:37485)
0809 22:45:41.632720 (+ 2181us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.632729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.632758 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:45:41.632855 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.632863 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.633079 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.634041 (+ 962us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:41.634287 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.635851 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.638414 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.639776 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.641588 (+ 1812us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.641838 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.641913 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.642085 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2015,"thread_start_us":1848,"threads_started":1}
I20260809 22:45:41.642616 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.630419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:38577 (local address 127.0.0.1:51860)
0809 22:45:41.632019 (+ 1600us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:41.632043 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:41.632149 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:41.633229 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:41.633237 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:41.633313 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:41.633884 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.633899 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.635620 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.635625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.638194 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.638211 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.639856 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.639862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.640174 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.640239 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 22:45:41.640369 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 22:45:41.641420 (+ 1051us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1474,"mutex_wait_us":961,"thread_start_us":1321,"threads_started":1}
I20260809 22:45:41.646209 26602 tablet_service.cc:1467] Processing CreateTablet for tablet 458c8002b188450986a492c6626e3a78 (DEFAULT_TABLE table=test-table [id=67117b87dda64990b4e1dc8133c7d355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:41.646224 26374 tablet_service.cc:1467] Processing CreateTablet for tablet 458c8002b188450986a492c6626e3a78 (DEFAULT_TABLE table=test-table [id=67117b87dda64990b4e1dc8133c7d355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:41.646222 26844 tablet_service.cc:1467] Processing CreateTablet for tablet 458c8002b188450986a492c6626e3a78 (DEFAULT_TABLE table=test-table [id=67117b87dda64990b4e1dc8133c7d355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:41.647636 26602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458c8002b188450986a492c6626e3a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:41.647687 26374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458c8002b188450986a492c6626e3a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:41.647691 26844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458c8002b188450986a492c6626e3a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:41.660693 26986 tablet_bootstrap.cc:492] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356: Bootstrap starting.
I20260809 22:45:41.662708 26987 tablet_bootstrap.cc:492] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca: Bootstrap starting.
I20260809 22:45:41.663586 26988 tablet_bootstrap.cc:492] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da: Bootstrap starting.
I20260809 22:45:41.666270 26986 tablet_bootstrap.cc:654] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:41.666714 26987 tablet_bootstrap.cc:654] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:41.667129 26988 tablet_bootstrap.cc:654] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:41.667697 26986 log.cc:824] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:41.667785 26987 log.cc:824] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:41.668366 26988 log.cc:824] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:41.671067 26986 tablet_bootstrap.cc:492] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356: No bootstrap required, opened a new log
I20260809 22:45:41.671373 26987 tablet_bootstrap.cc:492] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca: No bootstrap required, opened a new log
I20260809 22:45:41.671389 26986 ts_tablet_manager.cc:1397] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260809 22:45:41.671609 26988 tablet_bootstrap.cc:492] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da: No bootstrap required, opened a new log
I20260809 22:45:41.671653 26987 ts_tablet_manager.cc:1397] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260809 22:45:41.671952 26988 ts_tablet_manager.cc:1397] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260809 22:45:41.680071 26987 raft_consensus.cc:348] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } }
I20260809 22:45:41.680536 26987 raft_consensus.cc:374] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:41.680634 26987 raft_consensus.cc:729] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb7c319dee14ff3b893e6666dff81ca, State: Initialized, Role: FOLLOWER
I20260809 22:45:41.681277 26987 consensus_queue.cc:260] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } }
I20260809 22:45:41.682384 26986 raft_consensus.cc:348] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } }
I20260809 22:45:41.682407 26988 raft_consensus.cc:348] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } }
I20260809 22:45:41.682850 26988 raft_consensus.cc:374] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:41.682850 26986 raft_consensus.cc:374] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:41.682943 26988 raft_consensus.cc:729] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8ece2e19534a8895610e4ac254a4da, State: Initialized, Role: FOLLOWER
I20260809 22:45:41.682943 26986 raft_consensus.cc:729] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db53f482487d4442a8dbc0ec8c495356, State: Initialized, Role: FOLLOWER
I20260809 22:45:41.683172 26987 ts_tablet_manager.cc:1428] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260809 22:45:41.683686 26986 consensus_queue.cc:260] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } }
I20260809 22:45:41.683960 26988 consensus_queue.cc:260] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } }
I20260809 22:45:41.684875 26734 heartbeater.cc:503] Master 127.24.136.62:45543 was elected leader, sending a full tablet report...
I20260809 22:45:41.686242 26986 ts_tablet_manager.cc:1428] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260809 22:45:41.687273 26960 heartbeater.cc:503] Master 127.24.136.62:45543 was elected leader, sending a full tablet report...
I20260809 22:45:41.688006 26988 ts_tablet_manager.cc:1428] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da: Time spent starting tablet: real 0.016s user 0.008s sys 0.006s
W20260809 22:45:41.737363 26486 tablet.cc:2369] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:45:41.763202 26995 raft_consensus.cc:482] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:41.763563 26995 raft_consensus.cc:504] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } }
I20260809 22:45:41.765478 26995 leader_election.cc:288] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abb7c319dee14ff3b893e6666dff81ca (127.24.136.1:37485), ca8ece2e19534a8895610e4ac254a4da (127.24.136.3:38527)
W20260809 22:45:41.767228 26735 tablet.cc:2369] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:45:41.773018 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.766016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:57679 (local address 127.24.136.1:37485)
0809 22:45:41.766413 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.766419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.766859 (+ 440us) server_negotiation.cc:407] Connection header received
0809 22:45:41.767073 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.767078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.767164 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.767265 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:41.768001 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.768855 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.770909 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.771886 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.772031 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.772090 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.772573 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.772759 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260809 22:45:41.773034 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.765765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:37485 (local address 127.24.136.2:57679)
0809 22:45:41.766703 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:41.766728 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:41.766907 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:41.767321 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:41.767326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:41.767353 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:41.767804 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.767818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.769006 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.769011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.770684 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.770706 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.772148 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.772154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.772370 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.772405 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:45:41.772511 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 22:45:41.772635 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":605,"threads_started":1}
I20260809 22:45:41.773138 26984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.766179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:49993 (local address 127.24.136.3:38527)
0809 22:45:41.766372 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.766378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.767063 (+ 685us) server_negotiation.cc:407] Connection header received
0809 22:45:41.767360 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.767366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.767444 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.767532 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:41.768349 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.769191 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.771128 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.772164 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.772338 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.772414 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.772743 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.772895 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 22:45:41.773252 26999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.765770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:38527 (local address 127.24.136.2:49993)
0809 22:45:41.766933 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:41.766966 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:41.767186 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:41.767708 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:41.767715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:41.767735 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:41.768160 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.768172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.769387 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.769395 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.770914 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:41.770929 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.772386 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.772391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:41.772561 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.772595 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:45:41.772689 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 22:45:41.772821 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"mutex_wait_us":33,"thread_start_us":865,"threads_started":1}
I20260809 22:45:41.774415 26878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458c8002b188450986a492c6626e3a78" candidate_uuid: "db53f482487d4442a8dbc0ec8c495356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8ece2e19534a8895610e4ac254a4da" is_pre_election: true
I20260809 22:45:41.774430 26409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458c8002b188450986a492c6626e3a78" candidate_uuid: "db53f482487d4442a8dbc0ec8c495356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb7c319dee14ff3b893e6666dff81ca" is_pre_election: true
I20260809 22:45:41.775048 26409 raft_consensus.cc:2454] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db53f482487d4442a8dbc0ec8c495356 in term 0.
I20260809 22:45:41.775048 26878 raft_consensus.cc:2454] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db53f482487d4442a8dbc0ec8c495356 in term 0.
I20260809 22:45:41.775815 26528 leader_election.cc:302] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca8ece2e19534a8895610e4ac254a4da, db53f482487d4442a8dbc0ec8c495356; no voters:
I20260809 22:45:41.776253 26995 raft_consensus.cc:2781] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:41.776389 26995 raft_consensus.cc:482] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:41.776492 26995 raft_consensus.cc:3037] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:41.778954 26995 raft_consensus.cc:504] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } }
I20260809 22:45:41.779631 26995 leader_election.cc:288] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [CANDIDATE]: Term 1 election: Requested vote from peers abb7c319dee14ff3b893e6666dff81ca (127.24.136.1:37485), ca8ece2e19534a8895610e4ac254a4da (127.24.136.3:38527)
I20260809 22:45:41.780104 26409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458c8002b188450986a492c6626e3a78" candidate_uuid: "db53f482487d4442a8dbc0ec8c495356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb7c319dee14ff3b893e6666dff81ca"
I20260809 22:45:41.780207 26878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458c8002b188450986a492c6626e3a78" candidate_uuid: "db53f482487d4442a8dbc0ec8c495356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8ece2e19534a8895610e4ac254a4da"
I20260809 22:45:41.780308 26409 raft_consensus.cc:3037] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:41.780479 26878 raft_consensus.cc:3037] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:41.782630 26409 raft_consensus.cc:2454] T 458c8002b188450986a492c6626e3a78 P abb7c319dee14ff3b893e6666dff81ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db53f482487d4442a8dbc0ec8c495356 in term 1.
I20260809 22:45:41.782784 26878 raft_consensus.cc:2454] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db53f482487d4442a8dbc0ec8c495356 in term 1.
I20260809 22:45:41.783135 26523 leader_election.cc:302] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abb7c319dee14ff3b893e6666dff81ca, db53f482487d4442a8dbc0ec8c495356; no voters:
I20260809 22:45:41.783490 26995 raft_consensus.cc:2781] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:41.783974 26995 raft_consensus.cc:686] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [term 1 LEADER]: Becoming Leader. State: Replica: db53f482487d4442a8dbc0ec8c495356, State: Running, Role: LEADER
I20260809 22:45:41.784523 26995 consensus_queue.cc:237] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } }
I20260809 22:45:41.789788 26142 catalog_manager.cc:5495] T 458c8002b188450986a492c6626e3a78 P db53f482487d4442a8dbc0ec8c495356 reported cstate change: term changed from 0 to 1, leader changed from <none> to db53f482487d4442a8dbc0ec8c495356 (127.24.136.2). New cstate: current_term: 1 leader_uuid: "db53f482487d4442a8dbc0ec8c495356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abb7c319dee14ff3b893e6666dff81ca" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 37485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db53f482487d4442a8dbc0ec8c495356" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 38577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca8ece2e19534a8895610e4ac254a4da" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 38527 } health_report { overall_health: UNKNOWN } } }
W20260809 22:45:41.804001 26961 tablet.cc:2369] T 458c8002b188450986a492c6626e3a78 P ca8ece2e19534a8895610e4ac254a4da: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 22:45:41 dist-test-slave-pww0 krb5kdc[26063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:45:41.847826 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.830309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.24.136.1:37485)
0809 22:45:41.830597 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.830605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.831072 (+ 467us) server_negotiation.cc:407] Connection header received
0809 22:45:41.831296 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.831301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.831375 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.831508 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:41.833499 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.834381 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.836344 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.836878 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.836992 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.841876 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:41.841894 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:41.841911 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:41.841953 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:41.844877 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.845629 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.845637 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.845640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.845788 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.846201 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.846205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.846207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.846591 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:41.846903 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.847231 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.847450 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
Aug 09 22:45:41 dist-test-slave-pww0 krb5kdc[26063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:45:41 dist-test-slave-pww0 krb5kdc[26063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:45:41.900077 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.886269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58476 (local address 127.24.136.1:37485)
0809 22:45:41.886489 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.886495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.886890 (+ 395us) server_negotiation.cc:407] Connection header received
0809 22:45:41.887162 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.887167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.887243 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.887340 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:41.888747 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.889535 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.890815 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.891251 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.891350 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.895165 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:41.895170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:41.895174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:41.895213 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:41.897870 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.898463 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.898467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.898470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.898608 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.898940 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.898943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.898945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.899205 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:41.899393 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.899639 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.899809 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 22:45:41 dist-test-slave-pww0 krb5kdc[26063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:45:41 dist-test-slave-pww0 krb5kdc[26063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:45:41.950748 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.936431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58484 (local address 127.24.136.1:37485)
0809 22:45:41.936671 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.936677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.936996 (+ 319us) server_negotiation.cc:407] Connection header received
0809 22:45:41.937183 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.937187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.937259 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.937351 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:41.938849 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.939611 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.940889 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.941327 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.941448 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.945353 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:41.945358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:41.945362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:41.945400 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:41.948061 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.948741 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.948745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.948747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.948893 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.949272 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.949276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.949278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.949585 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:41.949864 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.950150 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.950421 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 22:45:41.952612 26339 tablet_service.cc:2908] Rejecting scan request for tablet 458c8002b188450986a492c6626e3a78: Uninitialized: clean time has not yet been initialized
I20260809 22:45:41.956274 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 26256
I20260809 22:45:41.969229 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:41.957129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.24.136.1:37485)
0809 22:45:41.957340 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:41.957346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:41.957604 (+ 258us) server_negotiation.cc:407] Connection header received
0809 22:45:41.957751 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:41.957755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:41.957831 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:41.957916 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:41.959066 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.959865 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.961068 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:41.961518 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:41.961630 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:41.963924 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:41.963929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:41.963933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:41.963967 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:41.966586 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.967300 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.967303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.967306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.967431 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:41.967815 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:41.967819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:41.967821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:41.968144 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:41.968381 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:41.968671 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 22:45:41.968882 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:45:41.970034 26478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:42.061094 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 26256
I20260809 22:45:42.089177 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 26498
Aug 09 22:45:42 dist-test-slave-pww0 krb5kdc[26063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:45:42.105517 26980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:42.090120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.24.136.2:38577)
0809 22:45:42.090407 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:42.090412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:42.090435 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:45:42.090498 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:42.090503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:42.090579 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:42.090694 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:42.091975 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:42.092757 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:42.094168 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:42.094633 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:42.094741 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:42.099133 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:42.099151 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:42.099169 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:42.099226 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:42.102265 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:42.103065 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:42.103073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:42.103076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:42.103254 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:42.103873 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:42.103878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:42.103880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:42.104278 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:42.104585 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:42.104920 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 22:45:42.105212 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 22:45:42.106318 26719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:42.198606 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 26498
W20260809 22:45:42.228067 26773 connection.cc:535] server connection from 127.24.136.2:49993 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 22:45:42.228368 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 26746
Aug 09 22:45:42 dist-test-slave-pww0 krb5kdc[26063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:45:42.243916 26984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:42.229137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.24.136.3:38527)
0809 22:45:42.229362 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:42.229368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:42.229389 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:45:42.229549 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:42.229554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:42.229622 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:42.229725 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:42.231047 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:42.231836 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:42.233217 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:42.233617 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:42.233738 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:42.238091 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:42.238108 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:42.238126 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:42.238167 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:42.240880 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:42.241621 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:42.241629 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:42.241631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:42.241787 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:42.242231 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:42.242234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:42.242236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:42.242633 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:42.242932 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:42.243305 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 22:45:42.243579 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:45:42.244652 26954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:42.334712 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 26746
I20260809 22:45:42.362735 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 26079
I20260809 22:45:42.364018 26205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:42.475956 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 26079
2026-08-09T22:45:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4573 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:45:42 dist-test-slave-pww0 krb5kdc[27032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:45:42 dist-test-slave-pww0 krb5kdc[27032](info): set up 2 sockets
Aug 09 22:45:42 dist-test-slave-pww0 krb5kdc[27032](info): commencing operation
krb5kdc: starting...
W20260809 22:45:43.721662 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:45:43 dist-test-slave-pww0 krb5kdc[27032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:45:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:45:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:45:43.939611 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:32925
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:32925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:44.284492 27056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:44.284868 27056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:44.285014 27056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:44.293020 27056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:45:44.293166 27056 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:45:44.293239 27056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:44.293334 27056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:44.293401 27056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:45:44.293486 27056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:45:44.304065 27056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:32925
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:32925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:44.305454 27056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:44.306870 27056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:44.319298 27069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:44.319387 27068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:44.319895 27074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:44.320252 27056 server_base.cc:1034] running on GCE node
I20260809 22:45:44.321195 27056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:44.322801 27056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:44.324034 27056 hybrid_clock.cc:648] HybridClock initialized: now 1786315544323961 us; error 55 us; skew 500 ppm
I20260809 22:45:44.324391 27056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:44 dist-test-slave-pww0 krb5kdc[27032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:44.327836 27056 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:45:44.329569 27056 webserver.cc:460] Webserver started at http://127.24.136.62:41151/ using document root <none> and password file <none>
I20260809 22:45:44.330367 27056 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:44.330485 27056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:44.330843 27056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:44.334159 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "d54a89bad376427f954adb21af663c39"
format_stamp: "Formatted at 2026-08-09 22:45:44 on dist-test-slave-pww0"
server_key: "b43b8ef945ba1acd195c0db6ec0736a2"
server_key_iv: "24c33541680ed3c7e9a403bcb7eaad71"
server_key_version: "encryptionkey@0"
I20260809 22:45:44.335018 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "d54a89bad376427f954adb21af663c39"
format_stamp: "Formatted at 2026-08-09 22:45:44 on dist-test-slave-pww0"
server_key: "b43b8ef945ba1acd195c0db6ec0736a2"
server_key_iv: "24c33541680ed3c7e9a403bcb7eaad71"
server_key_version: "encryptionkey@0"
I20260809 22:45:44.341015 27056 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 22:45:44.345188 27084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:44.346817 27056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:45:44.347036 27056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "d54a89bad376427f954adb21af663c39"
format_stamp: "Formatted at 2026-08-09 22:45:44 on dist-test-slave-pww0"
server_key: "b43b8ef945ba1acd195c0db6ec0736a2"
server_key_iv: "24c33541680ed3c7e9a403bcb7eaad71"
server_key_version: "encryptionkey@0"
I20260809 22:45:44.347289 27056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:44.374920 27056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:44.379845 27056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:44.380152 27056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:44.402747 27056 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:32925
I20260809 22:45:44.402762 27201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:32925 every 8 connection(s)
I20260809 22:45:44.404286 27056 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:45:44.408601 27203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:44.412009 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 27056
I20260809 22:45:44.412284 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:45:44.413098 25120 external_mini_cluster.cc:1428] Setting key 9e11a4d36f9030e73376279cc62d1c88
I20260809 22:45:44.422856 27203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39: Bootstrap starting.
I20260809 22:45:44.427174 27203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:44.428680 27203 log.cc:824] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:45:44 dist-test-slave-pww0 krb5kdc[27032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:44.431965 27203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39: No bootstrap required, opened a new log
I20260809 22:45:44.438345 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:44.415692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42522 (local address 127.24.136.62:32925)
0809 22:45:44.416580 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:44.416593 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:44.416640 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:45:44.418452 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:44.418480 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:44.418974 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:44.419387 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:44.421257 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:44.422652 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:44.424548 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:44.425380 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:44.425544 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:44.430693 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:44.430701 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:44.430718 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:44.430758 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:44.433890 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:44.434683 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:44.434691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:44.434699 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:44.434902 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:44.435315 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:44.435318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:44.435324 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:44.435984 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:44.436333 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:44.436781 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 22:45:44.437246 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":395,"threads_started":1}
I20260809 22:45:44.439986 27203 raft_consensus.cc:348] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54a89bad376427f954adb21af663c39" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32925 } }
I20260809 22:45:44.440361 27203 raft_consensus.cc:374] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:44.440476 27203 raft_consensus.cc:729] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d54a89bad376427f954adb21af663c39, State: Initialized, Role: FOLLOWER
I20260809 22:45:44.441066 27203 consensus_queue.cc:260] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54a89bad376427f954adb21af663c39" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32925 } }
I20260809 22:45:44.441346 27203 raft_consensus.cc:388] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:45:44.441555 27203 raft_consensus.cc:482] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:45:44.441723 27203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:44.444125 27203 raft_consensus.cc:504] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54a89bad376427f954adb21af663c39" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32925 } }
I20260809 22:45:44.444967 27203 leader_election.cc:302] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d54a89bad376427f954adb21af663c39; no voters:
I20260809 22:45:44.445827 27203 leader_election.cc:288] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:45:44.445941 27215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:44.447503 27215 raft_consensus.cc:686] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [term 1 LEADER]: Becoming Leader. State: Replica: d54a89bad376427f954adb21af663c39, State: Running, Role: LEADER
I20260809 22:45:44.448180 27203 sys_catalog.cc:564] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:45:44.448052 27215 consensus_queue.cc:237] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54a89bad376427f954adb21af663c39" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32925 } }
I20260809 22:45:44.450796 27217 sys_catalog.cc:455] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d54a89bad376427f954adb21af663c39. Latest consensus state: current_term: 1 leader_uuid: "d54a89bad376427f954adb21af663c39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54a89bad376427f954adb21af663c39" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32925 } } }
I20260809 22:45:44.451241 27217 sys_catalog.cc:458] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:44.452070 27216 sys_catalog.cc:455] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d54a89bad376427f954adb21af663c39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d54a89bad376427f954adb21af663c39" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32925 } } }
I20260809 22:45:44.452410 27216 sys_catalog.cc:458] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39 [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:44.455030 27223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:45:44.459940 27223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:45:44.470546 27223 catalog_manager.cc:1347] Generated new cluster ID: 675e1e236d7c4f12bed25e741250625a
I20260809 22:45:44.470681 27223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:45:44.488754 27223 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:45:44.490214 27223 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:45:44.512097 27223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39: Generated new TSK 0
I20260809 22:45:44.513127 27223 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:45:44.567679 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:32925
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:44.922286 27247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:44.922607 27247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:44.922765 27247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:44.930344 27247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:44.930503 27247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:44.930645 27247 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:45:44.930744 27247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:45:44.941027 27247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.136.62:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:44.942368 27247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:44.943819 27247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:44.957787 27257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:44.957821 27259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:44.958025 27256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:44.958935 27247 server_base.cc:1034] running on GCE node
I20260809 22:45:44.959774 27247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:44.960853 27247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:44.962110 27247 hybrid_clock.cc:648] HybridClock initialized: now 1786315544962027 us; error 56 us; skew 500 ppm
I20260809 22:45:44.962566 27247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:44 dist-test-slave-pww0 krb5kdc[27032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:44.969237 27247 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:45:44.971163 27247 webserver.cc:460] Webserver started at http://127.24.136.1:40181/ using document root <none> and password file <none>
I20260809 22:45:44.972059 27247 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:44.972179 27247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:44.972550 27247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:44.975871 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "bc8178dddac146fa9bf1a92bdca2ca99"
format_stamp: "Formatted at 2026-08-09 22:45:44 on dist-test-slave-pww0"
server_key: "ae6f071aabb3ee3b1398f98453492d19"
server_key_iv: "71c29f55dba039c693636e7fb8adfd19"
server_key_version: "encryptionkey@0"
I20260809 22:45:44.976814 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "bc8178dddac146fa9bf1a92bdca2ca99"
format_stamp: "Formatted at 2026-08-09 22:45:44 on dist-test-slave-pww0"
server_key: "ae6f071aabb3ee3b1398f98453492d19"
server_key_iv: "71c29f55dba039c693636e7fb8adfd19"
server_key_version: "encryptionkey@0"
I20260809 22:45:44.983529 27247 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260809 22:45:44.987906 27274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:44.989557 27247 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260809 22:45:44.989770 27247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "bc8178dddac146fa9bf1a92bdca2ca99"
format_stamp: "Formatted at 2026-08-09 22:45:44 on dist-test-slave-pww0"
server_key: "ae6f071aabb3ee3b1398f98453492d19"
server_key_iv: "71c29f55dba039c693636e7fb8adfd19"
server_key_version: "encryptionkey@0"
I20260809 22:45:44.990026 27247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:45.010838 27247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:45.015704 27247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:45.015991 27247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:45.018985 27247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:45.019143 27247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:45.019320 27247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:45.019392 27247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:45:45 dist-test-slave-pww0 krb5kdc[27032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:45.045254 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.019904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:52043 (local address 127.24.136.62:32925)
0809 22:45:45.020396 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:45.020401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:45.021776 (+ 1375us) server_negotiation.cc:407] Connection header received
0809 22:45:45.024178 (+ 2402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:45.024184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:45.024284 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:45.024410 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:45.027553 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.028522 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.031181 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.031711 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.031870 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.037418 (+ 5548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:45.037423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:45.037427 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:45.037468 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:45.040787 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.042160 (+ 1373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.042166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.042170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.042364 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.043024 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.043041 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.043045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.043498 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:45.043662 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:45.044518 (+ 856us) server_negotiation.cc:299] Negotiation successful
0809 22:45:45.044812 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":257,"threads_started":1}
I20260809 22:45:45.046090 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.020606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:32925 (local address 127.24.136.1:52043)
0809 22:45:45.021509 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:45.021615 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:45.023844 (+ 2229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:45.024822 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:45.024836 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:45.025940 (+ 1104us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:45.027198 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.027243 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.028840 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.028851 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.030925 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.030947 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.032033 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.032042 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.032379 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.033407 (+ 1028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:45.033449 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:45.037063 (+ 3614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:45.041029 (+ 3966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.041041 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.041069 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.041973 (+ 904us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.042537 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.042541 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.042547 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.042859 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.043686 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:45.043700 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:45.044116 (+ 416us) client_negotiation.cc:769] Sending connection context
0809 22:45:45.044370 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 22:45:45.044821 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":387,"threads_started":1}
I20260809 22:45:45.071139 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.054267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:60477 (local address 127.24.136.62:32925)
0809 22:45:45.054453 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:45.054458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:45.054527 (+ 69us) server_negotiation.cc:407] Connection header received
0809 22:45:45.054765 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:45.054769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:45.054843 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:45.054931 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:45.056337 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.057147 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.059214 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.059851 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.059981 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.064277 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:45.064283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:45.064286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:45.064326 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:45.067535 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.068705 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.068711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.068715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.068910 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.069409 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.069415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.069419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.069848 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:45.070018 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:45.070520 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 22:45:45.070774 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:45:45.071229 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.053664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:32925 (local address 127.24.136.1:60477)
0809 22:45:45.054418 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:45.054442 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:45.054626 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:45.054989 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:45.054994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:45.055706 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:45.056131 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.056146 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.057287 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.057294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.059016 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.059034 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.061351 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.061361 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.061601 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.062455 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:45.062483 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:45.063960 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:45.067760 (+ 3800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.067770 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.067774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.068486 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.069073 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.069079 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.069082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.069275 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.069995 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:45.070000 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:45.070210 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 22:45:45.070374 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 22:45:45.070754 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":653,"threads_started":1}
I20260809 22:45:45.074313 27247 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:39309
I20260809 22:45:45.074332 27494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:39309 every 8 connection(s)
I20260809 22:45:45.075954 27247 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:45:45.078235 27499 heartbeater.cc:347] Connected to a master server at 127.24.136.62:32925
I20260809 22:45:45.078474 27499 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:45.079178 27499 heartbeater.cc:511] Master 127.24.136.62:32925 requested a full tablet report, sending...
I20260809 22:45:45.081321 27120 ts_manager.cc:194] Registered new tserver with Master: bc8178dddac146fa9bf1a92bdca2ca99 (127.24.136.1:39309)
I20260809 22:45:45.082887 27120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:52043
I20260809 22:45:45.083951 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 27247
I20260809 22:45:45.084147 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:45:45.084790 25120 external_mini_cluster.cc:1428] Setting key 84452d308199c41139b2d3ae79630733
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:45:45.134608 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:32925
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:45.483544 27504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:45.483948 27504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:45.484112 27504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:45.492002 27504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:45.492168 27504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:45.492303 27504 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:45:45.492520 27504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:45:45.503199 27504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.136.62:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:45.504658 27504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:45.506126 27504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:45.520040 27512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:45.520082 27517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:45.520123 27519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:45.520983 27504 server_base.cc:1034] running on GCE node
I20260809 22:45:45.521658 27504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:45.522702 27504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:45.523975 27504 hybrid_clock.cc:648] HybridClock initialized: now 1786315545523904 us; error 40 us; skew 500 ppm
I20260809 22:45:45.524333 27504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:45 dist-test-slave-pww0 krb5kdc[27032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315545, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:45.527920 27504 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:45:45.529634 27504 webserver.cc:460] Webserver started at http://127.24.136.2:42943/ using document root <none> and password file <none>
I20260809 22:45:45.530424 27504 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:45.530539 27504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:45.530889 27504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:45.534129 27504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "3a384347a935475b884c14f971c4e301"
format_stamp: "Formatted at 2026-08-09 22:45:45 on dist-test-slave-pww0"
server_key: "e961869e8c6f44a80b13da3e48a75530"
server_key_iv: "e88f7fe9add5d67bcecd9fe0d0555753"
server_key_version: "encryptionkey@0"
I20260809 22:45:45.535058 27504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "3a384347a935475b884c14f971c4e301"
format_stamp: "Formatted at 2026-08-09 22:45:45 on dist-test-slave-pww0"
server_key: "e961869e8c6f44a80b13da3e48a75530"
server_key_iv: "e88f7fe9add5d67bcecd9fe0d0555753"
server_key_version: "encryptionkey@0"
I20260809 22:45:45.541028 27504 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.002s
I20260809 22:45:45.545022 27527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:45.546568 27504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:45:45.546761 27504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "3a384347a935475b884c14f971c4e301"
format_stamp: "Formatted at 2026-08-09 22:45:45 on dist-test-slave-pww0"
server_key: "e961869e8c6f44a80b13da3e48a75530"
server_key_iv: "e88f7fe9add5d67bcecd9fe0d0555753"
server_key_version: "encryptionkey@0"
I20260809 22:45:45.546991 27504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:45.568723 27504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:45.573729 27504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:45.574015 27504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:45.576967 27504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:45.577067 27504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:45.577217 27504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:45.577279 27504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:45:45 dist-test-slave-pww0 krb5kdc[27032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315545, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:45.602774 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.578163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:38829 (local address 127.24.136.62:32925)
0809 22:45:45.578598 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:45.578603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:45.580048 (+ 1445us) server_negotiation.cc:407] Connection header received
0809 22:45:45.582215 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:45.582221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:45.582315 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:45.582434 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:45.585192 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.586221 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.588384 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.589046 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.589205 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.594789 (+ 5584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:45.594796 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:45.594799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:45.594841 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:45.598423 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.599760 (+ 1337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.599767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.599771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.599964 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.600655 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.600661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.600665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.601067 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:45.601241 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:45.602100 (+ 859us) server_negotiation.cc:299] Negotiation successful
0809 22:45:45.602385 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":224,"threads_started":1}
I20260809 22:45:45.603974 27542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.578819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:32925 (local address 127.24.136.2:38829)
0809 22:45:45.579834 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:45.579887 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:45.581866 (+ 1979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:45.582955 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:45.582966 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:45.583961 (+ 995us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:45.584939 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.584971 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.586409 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.586414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.588163 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.588190 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.589293 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.589299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.589641 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.590606 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:45.590654 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:45.594383 (+ 3729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:45.598659 (+ 4276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.598670 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.598696 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.599556 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.600115 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.600121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.600123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.600504 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.601232 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:45.601241 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:45.601673 (+ 432us) client_negotiation.cc:769] Sending connection context
0809 22:45:45.601956 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 22:45:45.602553 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":445,"threads_started":1}
I20260809 22:45:45.625121 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.610103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:33073 (local address 127.24.136.62:32925)
0809 22:45:45.610325 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:45.610332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:45.611140 (+ 808us) server_negotiation.cc:407] Connection header received
0809 22:45:45.611324 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:45.611330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:45.611413 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:45.611582 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:45.613116 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.613876 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.615299 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.615917 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.616038 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.618701 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:45.618708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:45.618713 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:45.618759 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:45.621894 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.622922 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.622928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.622931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.623107 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.623652 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.623658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.623661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.624031 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:45.624157 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:45.624695 (+ 538us) server_negotiation.cc:299] Negotiation successful
0809 22:45:45.624886 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:45:45.625530 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.610165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:32925 (local address 127.24.136.2:33073)
0809 22:45:45.611020 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:45.611037 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:45.611186 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:45.611614 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:45.611620 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:45.612428 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:45.612938 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.612954 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.614052 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.614058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.615126 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.615140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.616150 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.616158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.616321 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.617068 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:45.617094 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:45.618428 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:45.622046 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.622053 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.622057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.622707 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.623286 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.623292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.623295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.623516 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.624177 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:45.624183 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:45.624435 (+ 252us) client_negotiation.cc:769] Sending connection context
0809 22:45:45.624616 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 22:45:45.624996 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":483,"threads_started":1}
I20260809 22:45:45.634514 27504 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:36767
I20260809 22:45:45.634552 27740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:36767 every 8 connection(s)
I20260809 22:45:45.635802 27504 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:45:45.637907 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 27504
I20260809 22:45:45.638151 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:45:45.638276 27743 heartbeater.cc:347] Connected to a master server at 127.24.136.62:32925
I20260809 22:45:45.638541 27743 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:45.638693 25120 external_mini_cluster.cc:1428] Setting key c34bacb4a6456e822139f014628d7f1a
I20260809 22:45:45.639322 27743 heartbeater.cc:511] Master 127.24.136.62:32925 requested a full tablet report, sending...
I20260809 22:45:45.641010 27120 ts_manager.cc:194] Registered new tserver with Master: 3a384347a935475b884c14f971c4e301 (127.24.136.2:36767)
I20260809 22:45:45.641857 27120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:38829
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:45:45.688180 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:32925
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:46.031787 27748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:46.032106 27748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:46.032262 27748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:46.039755 27748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:46.039897 27748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:46.040026 27748 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:45:46.040140 27748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:45:46.050854 27748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.136.62:32925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:46.052345 27748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:46.053807 27748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:46.067304 27754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:46.067389 27761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:46.067440 27755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:46.068997 27748 server_base.cc:1034] running on GCE node
I20260809 22:45:46.069694 27748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:46.070750 27748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:46.071997 27748 hybrid_clock.cc:648] HybridClock initialized: now 1786315546071931 us; error 44 us; skew 500 ppm
I20260809 22:45:46.072324 27748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:46 dist-test-slave-pww0 krb5kdc[27032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:46.075908 27748 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:45:46.079278 27748 webserver.cc:460] Webserver started at http://127.24.136.3:33643/ using document root <none> and password file <none>
I20260809 22:45:46.080101 27748 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:46.080219 27748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:46.080557 27748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:46.083750 27748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "a0a4f9587a2c4c17890f0b4de235bc76"
format_stamp: "Formatted at 2026-08-09 22:45:46 on dist-test-slave-pww0"
server_key: "375692d2425005b2adc6e8bf25950105"
server_key_iv: "8dee9d3af6422861095da804dcd3be67"
server_key_version: "encryptionkey@0"
I20260809 22:45:46.084617 27748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "a0a4f9587a2c4c17890f0b4de235bc76"
format_stamp: "Formatted at 2026-08-09 22:45:46 on dist-test-slave-pww0"
server_key: "375692d2425005b2adc6e8bf25950105"
server_key_iv: "8dee9d3af6422861095da804dcd3be67"
server_key_version: "encryptionkey@0"
I20260809 22:45:46.086262 27499 heartbeater.cc:503] Master 127.24.136.62:32925 was elected leader, sending a full tablet report...
I20260809 22:45:46.090668 27748 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 22:45:46.094828 27769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:46.096441 27748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:45:46.096643 27748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "a0a4f9587a2c4c17890f0b4de235bc76"
format_stamp: "Formatted at 2026-08-09 22:45:46 on dist-test-slave-pww0"
server_key: "375692d2425005b2adc6e8bf25950105"
server_key_iv: "8dee9d3af6422861095da804dcd3be67"
server_key_version: "encryptionkey@0"
I20260809 22:45:46.096871 27748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:46.117657 27748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:46.122380 27748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:46.122643 27748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:46.125741 27748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:46.125857 27748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:46.125999 27748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:46.126052 27748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:45:46 dist-test-slave-pww0 krb5kdc[27032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:46.152094 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.126890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:55347 (local address 127.24.136.62:32925)
0809 22:45:46.127419 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.127427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.128375 (+ 948us) server_negotiation.cc:407] Connection header received
0809 22:45:46.130883 (+ 2508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.130890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.130994 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.131142 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:46.133569 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.134525 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.137614 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.138095 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.138233 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.144020 (+ 5787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:46.144027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:46.144030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:46.144074 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:46.147088 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:46.148522 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:46.148530 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:46.148533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:46.148717 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:46.149509 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:46.149516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:46.149520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:46.150011 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:46.150232 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.151343 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.151685 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":255,"threads_started":1}
I20260809 22:45:46.153395 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.127170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:32925 (local address 127.24.136.3:55347)
0809 22:45:46.128184 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:46.128231 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:46.130473 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:46.131482 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:46.131496 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:46.132334 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:46.133340 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.133373 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.134740 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.134747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.137395 (+ 2648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.137418 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.138381 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.138389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.138752 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.139725 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:46.139763 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:46.143622 (+ 3859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:46.147326 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:46.147339 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:46.147365 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:46.148311 (+ 946us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:46.148884 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:46.148888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:46.148891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:46.149322 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:46.150233 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:46.150243 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:46.150811 (+ 568us) client_negotiation.cc:769] Sending connection context
0809 22:45:46.151207 (+ 396us) client_negotiation.cc:240] Negotiation successful
0809 22:45:46.151898 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":427,"threads_started":1}
I20260809 22:45:46.179607 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.162544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:38467 (local address 127.24.136.62:32925)
0809 22:45:46.162781 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.162788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.163432 (+ 644us) server_negotiation.cc:407] Connection header received
0809 22:45:46.163720 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.163725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.163823 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.163935 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:46.165468 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.166375 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.168500 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.169216 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.169382 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.172357 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:46.172364 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:46.172369 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:46.172419 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:46.176246 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:46.177402 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:46.177407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:46.177410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:46.177572 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:46.178007 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:46.178011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:46.178013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:46.178320 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:46.178469 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.178947 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.179192 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 22:45:46.179688 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.162503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:32925 (local address 127.24.136.3:38467)
0809 22:45:46.163279 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:46.163308 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:46.163555 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:46.164001 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:46.164010 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:46.164773 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:46.165268 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.165283 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.166596 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.166602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.168289 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.168310 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.169701 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.169708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.169931 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.170835 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:46.170865 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:46.172077 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:46.176472 (+ 4395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:46.176481 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:46.176485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:46.177215 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:46.177718 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:46.177722 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:46.177725 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:46.177871 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:46.178462 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:46.178467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:46.178663 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 22:45:46.178788 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 22:45:46.179173 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":393,"threads_started":1}
I20260809 22:45:46.190420 27748 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:40117
I20260809 22:45:46.190459 27999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:40117 every 8 connection(s)
I20260809 22:45:46.192090 27748 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:45:46.195218 28001 heartbeater.cc:347] Connected to a master server at 127.24.136.62:32925
I20260809 22:45:46.195613 28001 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:46.196451 28001 heartbeater.cc:511] Master 127.24.136.62:32925 requested a full tablet report, sending...
I20260809 22:45:46.198905 27120 ts_manager.cc:194] Registered new tserver with Master: a0a4f9587a2c4c17890f0b4de235bc76 (127.24.136.3:40117)
I20260809 22:45:46.199885 27120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:55347
I20260809 22:45:46.201606 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 27748
I20260809 22:45:46.201815 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:45:46.202433 25120 external_mini_cluster.cc:1428] Setting key 1d7cb8f8687a2f9887ecc2950fbf2b2f
I20260809 22:45:46.206524 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:45:46 dist-test-slave-pww0 krb5kdc[27032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:45:46 dist-test-slave-pww0 krb5kdc[27032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:46.263643 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.247888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42534 (local address 127.24.136.62:32925)
0809 22:45:46.248137 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.248144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.248496 (+ 352us) server_negotiation.cc:407] Connection header received
0809 22:45:46.248732 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.248738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.248828 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.248921 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:46.250559 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.251363 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.252833 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.253294 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.253444 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.257966 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:46.257972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:46.257975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:46.258022 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:46.260689 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:46.261472 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:46.261477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:46.261479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:46.261637 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:46.262099 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:46.262103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:46.262106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:46.262429 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:46.262685 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.263042 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.263283 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:45:46.268309 27120 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:45:46.275777 27120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:46.300314 28027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.288853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39309 (local address 127.0.0.1:48334)
0809 22:45:46.291000 (+ 2147us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:46.291020 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:46.291199 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:46.292297 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:46.292306 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:46.292340 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:46.292772 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.292785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.294794 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.294800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.297260 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.297275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.299163 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.299170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.299393 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.299435 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:45:46.299596 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 22:45:46.299809 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2023,"mutex_wait_us":1,"spinlock_wait_cycles":9728,"thread_start_us":1992,"threads_started":1}
I20260809 22:45:46.301513 28029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.289299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48334 (local address 127.24.136.1:39309)
0809 22:45:46.289970 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.289981 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.291325 (+ 1344us) server_negotiation.cc:407] Connection header received
0809 22:45:46.291476 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.291486 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.291753 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.291957 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:46.292983 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.294670 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.297466 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.298942 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.299069 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.299304 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.299681 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.301128 (+ 1447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":377,"threads_started":1}
I20260809 22:45:46.301777 28028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.289004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47488 (local address 127.24.136.3:40117)
0809 22:45:46.290578 (+ 1574us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.290586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.291215 (+ 629us) server_negotiation.cc:407] Connection header received
0809 22:45:46.291479 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.291487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.293121 (+ 1634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.293324 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:46.294912 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.296413 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.298790 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.300330 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.300457 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.300695 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.301241 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.301421 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1397,"thread_start_us":1161,"threads_started":1}
I20260809 22:45:46.302496 28031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.291792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48952 (local address 127.24.136.2:36767)
0809 22:45:46.294026 (+ 2234us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.294036 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.294697 (+ 661us) server_negotiation.cc:407] Connection header received
0809 22:45:46.294914 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.294923 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.295179 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.295367 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:46.296327 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.297678 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.299787 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.301109 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.301248 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.301440 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.302012 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.302188 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2065,"thread_start_us":1896,"threads_started":1}
I20260809 22:45:46.302547 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.291375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:36767 (local address 127.0.0.1:48952)
0809 22:45:46.294570 (+ 3195us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:46.294590 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:46.294714 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:46.295649 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:46.295654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:46.295684 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:46.296118 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.296131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.297820 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.297825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.299599 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.299613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.301407 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.301413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.301704 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.301741 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:45:46.301828 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 22:45:46.302120 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3035,"mutex_wait_us":21,"thread_start_us":2880,"threads_started":1}
I20260809 22:45:46.303031 28025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.288704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:40117 (local address 127.0.0.1:47488)
0809 22:45:46.290934 (+ 2230us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:46.290986 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:46.291201 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:46.293969 (+ 2768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:46.293974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:46.294001 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:46.294586 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.294600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.296584 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.296589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.298577 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.298593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.300844 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.300851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.301019 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.301053 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:45:46.301991 (+ 938us) client_negotiation.cc:240] Negotiation successful
0809 22:45:46.302143 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1945,"mutex_wait_us":27,"spinlock_wait_cycles":3968,"thread_start_us":1798,"threads_started":1}
I20260809 22:45:46.306087 27607 tablet_service.cc:1467] Processing CreateTablet for tablet 1dea4dc5b9814c48b198fbdf5a46ad11 (DEFAULT_TABLE table=test-table [id=3ad5a85a776540d48b79de76d60927f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:46.306088 27367 tablet_service.cc:1467] Processing CreateTablet for tablet 1dea4dc5b9814c48b198fbdf5a46ad11 (DEFAULT_TABLE table=test-table [id=3ad5a85a776540d48b79de76d60927f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:46.306720 27876 tablet_service.cc:1467] Processing CreateTablet for tablet 1dea4dc5b9814c48b198fbdf5a46ad11 (DEFAULT_TABLE table=test-table [id=3ad5a85a776540d48b79de76d60927f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:46.307607 27607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dea4dc5b9814c48b198fbdf5a46ad11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:46.307785 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dea4dc5b9814c48b198fbdf5a46ad11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:46.308357 27876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dea4dc5b9814c48b198fbdf5a46ad11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:46.321650 28040 tablet_bootstrap.cc:492] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: Bootstrap starting.
I20260809 22:45:46.323948 28042 tablet_bootstrap.cc:492] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: Bootstrap starting.
I20260809 22:45:46.325793 28041 tablet_bootstrap.cc:492] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: Bootstrap starting.
I20260809 22:45:46.326028 28040 tablet_bootstrap.cc:654] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:46.327556 28040 log.cc:824] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:46.327852 28042 tablet_bootstrap.cc:654] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:46.329396 28042 log.cc:824] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:46.329555 28041 tablet_bootstrap.cc:654] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:46.330869 28041 log.cc:824] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:46.333997 28041 tablet_bootstrap.cc:492] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: No bootstrap required, opened a new log
I20260809 22:45:46.334311 28041 ts_tablet_manager.cc:1397] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:45:46.334384 28042 tablet_bootstrap.cc:492] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: No bootstrap required, opened a new log
I20260809 22:45:46.334384 28040 tablet_bootstrap.cc:492] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: No bootstrap required, opened a new log
I20260809 22:45:46.334681 28040 ts_tablet_manager.cc:1397] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260809 22:45:46.334678 28042 ts_tablet_manager.cc:1397] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260809 22:45:46.342674 28041 raft_consensus.cc:348] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } }
I20260809 22:45:46.343128 28041 raft_consensus.cc:374] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:46.343253 28041 raft_consensus.cc:729] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8178dddac146fa9bf1a92bdca2ca99, State: Initialized, Role: FOLLOWER
I20260809 22:45:46.344048 28041 consensus_queue.cc:260] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } }
I20260809 22:45:46.345992 28041 ts_tablet_manager.cc:1428] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260809 22:45:46.345891 28040 raft_consensus.cc:348] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } }
I20260809 22:45:46.345889 28042 raft_consensus.cc:348] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } }
I20260809 22:45:46.346374 28040 raft_consensus.cc:374] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:46.346374 28042 raft_consensus.cc:374] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:46.346493 28042 raft_consensus.cc:729] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a4f9587a2c4c17890f0b4de235bc76, State: Initialized, Role: FOLLOWER
I20260809 22:45:46.346485 28040 raft_consensus.cc:729] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a384347a935475b884c14f971c4e301, State: Initialized, Role: FOLLOWER
I20260809 22:45:46.347168 28040 consensus_queue.cc:260] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } }
I20260809 22:45:46.347210 28042 consensus_queue.cc:260] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } }
I20260809 22:45:46.348975 27743 heartbeater.cc:503] Master 127.24.136.62:32925 was elected leader, sending a full tablet report...
I20260809 22:45:46.349445 28042 ts_tablet_manager.cc:1428] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260809 22:45:46.349794 28040 ts_tablet_manager.cc:1428] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260809 22:45:46.350447 28001 heartbeater.cc:503] Master 127.24.136.62:32925 was elected leader, sending a full tablet report...
W20260809 22:45:46.387986 27744 tablet.cc:2369] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:45:46.444070 28003 tablet.cc:2369] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:45:46.578826 27500 tablet.cc:2369] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:45:46.600219 28058 raft_consensus.cc:482] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:46.600587 28058 raft_consensus.cc:504] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } }
I20260809 22:45:46.602553 28058 leader_election.cc:288] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0a4f9587a2c4c17890f0b4de235bc76 (127.24.136.3:40117), bc8178dddac146fa9bf1a92bdca2ca99 (127.24.136.1:39309)
I20260809 22:45:46.611959 28029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.603035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:45443 (local address 127.24.136.1:39309)
0809 22:45:46.603609 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.603616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.604146 (+ 530us) server_negotiation.cc:407] Connection header received
0809 22:45:46.604293 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.604298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.604377 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.604467 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:46.605217 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.606289 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.609320 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.610587 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.610760 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.610835 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.611567 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.611715 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382}
I20260809 22:45:46.612004 28028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.603047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:44057 (local address 127.24.136.3:40117)
0809 22:45:46.603376 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.603383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.603891 (+ 508us) server_negotiation.cc:407] Connection header received
0809 22:45:46.604113 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.604120 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.604210 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.604322 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:46.605096 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.606299 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.609320 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.610572 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.610755 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.610835 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.611621 (+ 786us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.611782 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260809 22:45:46.612051 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.603079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39309 (local address 127.24.136.2:45443)
0809 22:45:46.604041 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:46.604056 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:46.604163 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:46.604512 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:46.604516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:46.604594 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:46.605047 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.605059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.606462 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.606468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.609106 (+ 2638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.609126 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.611021 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.611034 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.611256 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.611380 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 22:45:46.611533 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 22:45:46.611649 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":54,"spinlock_wait_cycles":768,"thread_start_us":652,"threads_started":1}
I20260809 22:45:46.612759 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.602860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:40117 (local address 127.24.136.2:44057)
0809 22:45:46.603731 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:46.603758 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:46.603960 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:46.604356 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:46.604359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:46.604381 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:46.604908 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.604925 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.606462 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.606468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.609110 (+ 2642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.609126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.611053 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.611061 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.611393 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.611435 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:45:46.612254 (+ 819us) client_negotiation.cc:240] Negotiation successful
0809 22:45:46.612399 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":170,"spinlock_wait_cycles":71808,"thread_start_us":457,"threads_started":1}
I20260809 22:45:46.613266 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dea4dc5b9814c48b198fbdf5a46ad11" candidate_uuid: "3a384347a935475b884c14f971c4e301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" is_pre_election: true
I20260809 22:45:46.613835 27402 raft_consensus.cc:2454] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a384347a935475b884c14f971c4e301 in term 0.
I20260809 22:45:46.613822 27920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dea4dc5b9814c48b198fbdf5a46ad11" candidate_uuid: "3a384347a935475b884c14f971c4e301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" is_pre_election: true
I20260809 22:45:46.614302 27920 raft_consensus.cc:2454] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a384347a935475b884c14f971c4e301 in term 0.
I20260809 22:45:46.614513 27530 leader_election.cc:302] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a384347a935475b884c14f971c4e301, bc8178dddac146fa9bf1a92bdca2ca99; no voters:
I20260809 22:45:46.614990 28058 raft_consensus.cc:2781] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:46.615130 28058 raft_consensus.cc:482] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:46.615236 28058 raft_consensus.cc:3037] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:46.617601 28058 raft_consensus.cc:504] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } }
I20260809 22:45:46.618299 28058 leader_election.cc:288] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [CANDIDATE]: Term 1 election: Requested vote from peers a0a4f9587a2c4c17890f0b4de235bc76 (127.24.136.3:40117), bc8178dddac146fa9bf1a92bdca2ca99 (127.24.136.1:39309)
I20260809 22:45:46.618995 27920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dea4dc5b9814c48b198fbdf5a46ad11" candidate_uuid: "3a384347a935475b884c14f971c4e301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4f9587a2c4c17890f0b4de235bc76"
I20260809 22:45:46.618995 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dea4dc5b9814c48b198fbdf5a46ad11" candidate_uuid: "3a384347a935475b884c14f971c4e301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8178dddac146fa9bf1a92bdca2ca99"
I20260809 22:45:46.619238 27920 raft_consensus.cc:3037] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:46.619246 27402 raft_consensus.cc:3037] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:46.623749 27402 raft_consensus.cc:2454] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a384347a935475b884c14f971c4e301 in term 1.
I20260809 22:45:46.623970 27920 raft_consensus.cc:2454] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a384347a935475b884c14f971c4e301 in term 1.
I20260809 22:45:46.624377 27530 leader_election.cc:302] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a384347a935475b884c14f971c4e301, bc8178dddac146fa9bf1a92bdca2ca99; no voters:
I20260809 22:45:46.624752 28058 raft_consensus.cc:2781] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:46.625300 28058 raft_consensus.cc:686] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 1 LEADER]: Becoming Leader. State: Replica: 3a384347a935475b884c14f971c4e301, State: Running, Role: LEADER
I20260809 22:45:46.625898 28058 consensus_queue.cc:237] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } }
I20260809 22:45:46.631976 27119 catalog_manager.cc:5495] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a384347a935475b884c14f971c4e301 (127.24.136.2). New cstate: current_term: 1 leader_uuid: "3a384347a935475b884c14f971c4e301" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } health_report { overall_health: HEALTHY } } }
I20260809 22:45:46.680750 28031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.674708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.24.136.2:36767)
0809 22:45:46.674933 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.674938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.675101 (+ 163us) server_negotiation.cc:407] Connection header received
0809 22:45:46.675417 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.675421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.675541 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.675685 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:46.676069 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.676811 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.678349 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.678785 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.678893 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.679525 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:46.680147 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:46.680278 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.680425 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.680505 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:45:46.689903 27402 raft_consensus.cc:1261] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [term 1 FOLLOWER]: Refusing update from remote peer 3a384347a935475b884c14f971c4e301: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:46.689898 27920 raft_consensus.cc:1261] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 FOLLOWER]: Refusing update from remote peer 3a384347a935475b884c14f971c4e301: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:46.691002 28058 consensus_queue.cc:1035] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:46.691332 28061 consensus_queue.cc:1035] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:46.704479 28068 mvcc.cc:205] Tried to move back new op lower bound from 7316748479230775296 to 7316748478984585216. Current Snapshot: MvccSnapshot[applied={T|T < 7316748479230775296}]
I20260809 22:45:46.708482 28070 mvcc.cc:205] Tried to move back new op lower bound from 7316748479230775296 to 7316748478984585216. Current Snapshot: MvccSnapshot[applied={T|T < 7316748479230775296}]
I20260809 22:45:46.724972 27120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:46.725553 27120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:46.730248 27120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39: Sending DeleteTablet for 3 replicas of tablet 1dea4dc5b9814c48b198fbdf5a46ad11
I20260809 22:45:46.731745 27876 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dea4dc5b9814c48b198fbdf5a46ad11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:46 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:47488
I20260809 22:45:46.731781 27607 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dea4dc5b9814c48b198fbdf5a46ad11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:46 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:48952
I20260809 22:45:46.732424 27367 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dea4dc5b9814c48b198fbdf5a46ad11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:46 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:48334
I20260809 22:45:46.735558 27120 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:45:46.736302 27120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:46.758060 27606 tablet_service.cc:1467] Processing CreateTablet for tablet d73454de97b849ff916534973ccacfdd (DEFAULT_TABLE table=test-table [id=2841ad565b594349a3f4b6466e4c0594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:46.758492 27874 tablet_service.cc:1467] Processing CreateTablet for tablet d73454de97b849ff916534973ccacfdd (DEFAULT_TABLE table=test-table [id=2841ad565b594349a3f4b6466e4c0594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:46.758879 27606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73454de97b849ff916534973ccacfdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:46.759132 27874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73454de97b849ff916534973ccacfdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:46.761584 28089 tablet_replica.cc:330] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: stopping tablet replica
I20260809 22:45:46.762161 28089 raft_consensus.cc:2229] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:46.762753 28089 raft_consensus.cc:2258] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:46.763792 28087 tablet_replica.cc:330] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: stopping tablet replica
I20260809 22:45:46.764153 27363 tablet_service.cc:1467] Processing CreateTablet for tablet d73454de97b849ff916534973ccacfdd (DEFAULT_TABLE table=test-table [id=2841ad565b594349a3f4b6466e4c0594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:46.764978 27363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73454de97b849ff916534973ccacfdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:46.765347 28090 tablet_replica.cc:330] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: stopping tablet replica
I20260809 22:45:46.765829 28090 raft_consensus.cc:2229] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:46.767627 28042 tablet_bootstrap.cc:492] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: Bootstrap starting.
I20260809 22:45:46.769109 28087 raft_consensus.cc:2229] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:45:46.769711 28042 tablet_bootstrap.cc:654] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:46.769987 28087 raft_consensus.cc:2258] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:46.770670 28090 raft_consensus.cc:2258] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:46.777375 28089 ts_tablet_manager.cc:1905] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:46.778223 28041 tablet_bootstrap.cc:492] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: Bootstrap starting.
I20260809 22:45:46.780344 28041 tablet_bootstrap.cc:654] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:46.782002 28090 ts_tablet_manager.cc:1905] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:46.783674 28087 ts_tablet_manager.cc:1905] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:46.791301 28040 tablet_bootstrap.cc:492] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: Bootstrap starting.
I20260809 22:45:46.793426 28040 tablet_bootstrap.cc:654] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:46.796443 28041 tablet_bootstrap.cc:492] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: No bootstrap required, opened a new log
I20260809 22:45:46.796666 28041 ts_tablet_manager.cc:1397] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.008s
I20260809 22:45:46.797609 28041 raft_consensus.cc:348] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:46.798005 28041 raft_consensus.cc:374] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:46.798152 28041 raft_consensus.cc:729] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8178dddac146fa9bf1a92bdca2ca99, State: Initialized, Role: FOLLOWER
I20260809 22:45:46.798470 28041 consensus_queue.cc:260] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:46.799136 28041 ts_tablet_manager.cc:1428] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:45:46.799691 28042 tablet_bootstrap.cc:492] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: No bootstrap required, opened a new log
I20260809 22:45:46.800446 28042 ts_tablet_manager.cc:1397] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.004s
I20260809 22:45:46.802497 28042 raft_consensus.cc:348] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:46.805622 28042 raft_consensus.cc:374] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:46.805842 28042 raft_consensus.cc:729] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a4f9587a2c4c17890f0b4de235bc76, State: Initialized, Role: FOLLOWER
I20260809 22:45:46.806270 28042 consensus_queue.cc:260] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:46.807730 28042 ts_tablet_manager.cc:1428] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260809 22:45:46.809068 28040 tablet_bootstrap.cc:492] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: No bootstrap required, opened a new log
I20260809 22:45:46.809253 28040 ts_tablet_manager.cc:1397] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.004s
I20260809 22:45:46.810140 28040 raft_consensus.cc:348] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:46.810431 28040 raft_consensus.cc:374] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:46.810547 28040 raft_consensus.cc:729] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a384347a935475b884c14f971c4e301, State: Initialized, Role: FOLLOWER
I20260809 22:45:46.810878 28040 consensus_queue.cc:260] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:46.813141 28040 ts_tablet_manager.cc:1428] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260809 22:45:46.846194 28087 ts_tablet_manager.cc:1918] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:45:46.846433 28087 log.cc:1196] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/1dea4dc5b9814c48b198fbdf5a46ad11
I20260809 22:45:46.847069 28087 ts_tablet_manager.cc:1939] T 1dea4dc5b9814c48b198fbdf5a46ad11 P 3a384347a935475b884c14f971c4e301: Deleting consensus metadata
I20260809 22:45:46.858310 28061 raft_consensus.cc:482] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:46.858572 28061 raft_consensus.cc:504] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:46.859319 28061 leader_election.cc:288] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc8178dddac146fa9bf1a92bdca2ca99 (127.24.136.1:39309), a0a4f9587a2c4c17890f0b4de235bc76 (127.24.136.3:40117)
I20260809 22:45:46.860002 27920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73454de97b849ff916534973ccacfdd" candidate_uuid: "3a384347a935475b884c14f971c4e301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" is_pre_election: true
I20260809 22:45:46.859993 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73454de97b849ff916534973ccacfdd" candidate_uuid: "3a384347a935475b884c14f971c4e301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" is_pre_election: true
I20260809 22:45:46.860237 27920 raft_consensus.cc:2454] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a384347a935475b884c14f971c4e301 in term 0.
I20260809 22:45:46.860260 27402 raft_consensus.cc:2454] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a384347a935475b884c14f971c4e301 in term 0.
I20260809 22:45:46.860702 27532 leader_election.cc:302] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a384347a935475b884c14f971c4e301, a0a4f9587a2c4c17890f0b4de235bc76; no voters:
I20260809 22:45:46.861040 28061 raft_consensus.cc:2781] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:46.861148 28061 raft_consensus.cc:482] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:46.861277 28061 raft_consensus.cc:3037] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:46.914086 27091 catalog_manager.cc:4841] TS 3a384347a935475b884c14f971c4e301 (127.24.136.2:36767): tablet 1dea4dc5b9814c48b198fbdf5a46ad11 (table test-table [id=3ad5a85a776540d48b79de76d60927f6]) successfully deleted
I20260809 22:45:46.950723 28090 ts_tablet_manager.cc:1918] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:45:46.951026 28090 log.cc:1196] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/1dea4dc5b9814c48b198fbdf5a46ad11
I20260809 22:45:46.950781 28089 ts_tablet_manager.cc:1918] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:45:46.951242 28089 log.cc:1196] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/1dea4dc5b9814c48b198fbdf5a46ad11
I20260809 22:45:46.951809 28090 ts_tablet_manager.cc:1939] T 1dea4dc5b9814c48b198fbdf5a46ad11 P bc8178dddac146fa9bf1a92bdca2ca99: Deleting consensus metadata
I20260809 22:45:46.952186 28089 ts_tablet_manager.cc:1939] T 1dea4dc5b9814c48b198fbdf5a46ad11 P a0a4f9587a2c4c17890f0b4de235bc76: Deleting consensus metadata
I20260809 22:45:46.952430 28061 raft_consensus.cc:504] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:46.953794 28061 leader_election.cc:288] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [CANDIDATE]: Term 1 election: Requested vote from peers bc8178dddac146fa9bf1a92bdca2ca99 (127.24.136.1:39309), a0a4f9587a2c4c17890f0b4de235bc76 (127.24.136.3:40117)
I20260809 22:45:46.953950 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73454de97b849ff916534973ccacfdd" candidate_uuid: "3a384347a935475b884c14f971c4e301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8178dddac146fa9bf1a92bdca2ca99"
I20260809 22:45:46.954191 27402 raft_consensus.cc:3037] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:46.954385 27920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73454de97b849ff916534973ccacfdd" candidate_uuid: "3a384347a935475b884c14f971c4e301" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a4f9587a2c4c17890f0b4de235bc76"
I20260809 22:45:46.954609 27920 raft_consensus.cc:3037] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:46.957793 27086 catalog_manager.cc:4841] TS bc8178dddac146fa9bf1a92bdca2ca99 (127.24.136.1:39309): tablet 1dea4dc5b9814c48b198fbdf5a46ad11 (table test-table [id=3ad5a85a776540d48b79de76d60927f6]) successfully deleted
I20260809 22:45:46.958562 27091 catalog_manager.cc:4841] TS a0a4f9587a2c4c17890f0b4de235bc76 (127.24.136.3:40117): tablet 1dea4dc5b9814c48b198fbdf5a46ad11 (table test-table [id=3ad5a85a776540d48b79de76d60927f6]) successfully deleted
I20260809 22:45:47.050801 27920 raft_consensus.cc:2454] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a384347a935475b884c14f971c4e301 in term 1.
I20260809 22:45:47.050793 27402 raft_consensus.cc:2454] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a384347a935475b884c14f971c4e301 in term 1.
I20260809 22:45:47.051635 27530 leader_election.cc:302] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a384347a935475b884c14f971c4e301, bc8178dddac146fa9bf1a92bdca2ca99; no voters:
I20260809 22:45:47.052176 28061 raft_consensus.cc:2781] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:47.052838 28061 raft_consensus.cc:686] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 1 LEADER]: Becoming Leader. State: Replica: 3a384347a935475b884c14f971c4e301, State: Running, Role: LEADER
I20260809 22:45:47.053176 28061 consensus_queue.cc:237] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.058535 27120 catalog_manager.cc:5495] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a384347a935475b884c14f971c4e301 (127.24.136.2). New cstate: current_term: 1 leader_uuid: "3a384347a935475b884c14f971c4e301" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } health_report { overall_health: UNKNOWN } } }
Aug 09 22:45:47 dist-test-slave-pww0 krb5kdc[27032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:47.137014 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.118288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:32925 (local address 127.0.0.1:42546)
0809 22:45:47.119358 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.119381 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.119606 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.120050 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.120059 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.120757 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:47.120972 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.120982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.122564 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.122570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.123661 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.123676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.124952 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.124960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.125186 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.126157 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:47.126184 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:47.129718 (+ 3534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:47.133865 (+ 4147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:47.133888 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:47.133899 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:47.134583 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:47.135091 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:47.135095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:47.135098 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:47.135262 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:47.136054 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:47.136061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:47.136257 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.136411 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.136677 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":627,"threads_started":1}
I20260809 22:45:47.137600 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.118326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.24.136.62:32925)
0809 22:45:47.119510 (+ 1184us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.119517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.119549 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:45:47.119763 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.119769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.119856 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.119999 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:47.121165 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.122380 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.123918 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.124598 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.124750 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.130005 (+ 5255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:47.130011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:47.130014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:47.130055 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:47.133632 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.134764 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.134769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.134773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.134942 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.135424 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.135429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.135432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.135889 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:47.136064 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.136598 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.136865 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":340,"threads_started":1}
I20260809 22:45:47.143852 27120 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:42546:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:45:47.145509 27120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:47.156524 27363 tablet_service.cc:1467] Processing CreateTablet for tablet 52ec633524c04f0f90e1714aff03f182 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cbe6f3ba2c994836ab3a805aa7b04482]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:45:47.156500 27606 tablet_service.cc:1467] Processing CreateTablet for tablet 52ec633524c04f0f90e1714aff03f182 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cbe6f3ba2c994836ab3a805aa7b04482]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:45:47.157131 27606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ec633524c04f0f90e1714aff03f182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:47.157121 27363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ec633524c04f0f90e1714aff03f182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:47.161986 27874 tablet_service.cc:1467] Processing CreateTablet for tablet 52ec633524c04f0f90e1714aff03f182 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cbe6f3ba2c994836ab3a805aa7b04482]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:45:47.162634 27874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52ec633524c04f0f90e1714aff03f182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:47.165771 28040 tablet_bootstrap.cc:492] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: Bootstrap starting.
I20260809 22:45:47.166692 28041 tablet_bootstrap.cc:492] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: Bootstrap starting.
I20260809 22:45:47.167627 28040 tablet_bootstrap.cc:654] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:47.168527 28041 tablet_bootstrap.cc:654] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:47.172107 28042 tablet_bootstrap.cc:492] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: Bootstrap starting.
I20260809 22:45:47.174614 28042 tablet_bootstrap.cc:654] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:47.188077 28040 tablet_bootstrap.cc:492] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: No bootstrap required, opened a new log
I20260809 22:45:47.188331 28040 ts_tablet_manager.cc:1397] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.011s
I20260809 22:45:47.188978 28041 tablet_bootstrap.cc:492] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: No bootstrap required, opened a new log
I20260809 22:45:47.189203 28041 ts_tablet_manager.cc:1397] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.008s
I20260809 22:45:47.189222 28042 tablet_bootstrap.cc:492] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: No bootstrap required, opened a new log
I20260809 22:45:47.189230 28040 raft_consensus.cc:348] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.189414 28042 ts_tablet_manager.cc:1397] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.003s
I20260809 22:45:47.189510 28040 raft_consensus.cc:374] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:47.189599 28040 raft_consensus.cc:729] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a384347a935475b884c14f971c4e301, State: Initialized, Role: FOLLOWER
I20260809 22:45:47.189993 28040 consensus_queue.cc:260] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.190032 28041 raft_consensus.cc:348] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.190311 28041 raft_consensus.cc:374] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:47.190399 28041 raft_consensus.cc:729] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8178dddac146fa9bf1a92bdca2ca99, State: Initialized, Role: FOLLOWER
I20260809 22:45:47.190603 28040 ts_tablet_manager.cc:1428] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 22:45:47.190680 28041 consensus_queue.cc:260] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.190665 28061 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.191007 28061 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:47.191190 28048 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.191524 28048 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:47.191990 28041 ts_tablet_manager.cc:1428] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 22:45:47.193084 28042 raft_consensus.cc:348] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.193439 28042 raft_consensus.cc:374] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:47.193542 28042 raft_consensus.cc:729] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a4f9587a2c4c17890f0b4de235bc76, State: Initialized, Role: FOLLOWER
I20260809 22:45:47.193935 28042 consensus_queue.cc:260] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.194388 28056 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.194790 28056 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:47.196967 28042 ts_tablet_manager.cc:1428] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: Time spent starting tablet: real 0.007s user 0.003s sys 0.001s
I20260809 22:45:47.354856 28056 raft_consensus.cc:482] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:47.355093 28056 raft_consensus.cc:504] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.356884 28056 leader_election.cc:288] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc8178dddac146fa9bf1a92bdca2ca99 (127.24.136.1:39309), 3a384347a935475b884c14f971c4e301 (127.24.136.2:36767)
I20260809 22:45:47.364748 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.357340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:40303 (local address 127.24.136.2:36767)
0809 22:45:47.358122 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.358129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.358153 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:45:47.358245 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.358250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.358356 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.358479 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:47.359145 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.360154 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.362593 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.363820 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.364023 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.364108 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.364376 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.364517 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":545,"threads_started":1}
I20260809 22:45:47.365200 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.357367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39309 (local address 127.24.136.3:33259)
0809 22:45:47.358358 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.358376 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.358496 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.359230 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.359234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.359259 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:47.359584 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.359594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.360821 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.360827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.362730 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.362741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.364413 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.364419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.364589 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.364625 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.364751 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.364882 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":708,"threads_started":1}
I20260809 22:45:47.365360 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.357811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:33259 (local address 127.24.136.1:39309)
0809 22:45:47.358903 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.358907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.358923 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:45:47.358982 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.358986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.359052 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.359161 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:47.359784 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.360661 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.362902 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.364160 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.364311 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.364386 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.364822 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.365073 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986,"thread_start_us":888,"threads_started":1}
I20260809 22:45:47.365248 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.357238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:36767 (local address 127.24.136.3:40303)
0809 22:45:47.357826 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.357853 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.358031 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.358548 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.358554 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.358612 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:47.358941 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.358952 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.360324 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.360328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.362383 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.362400 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.364020 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.364025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.364178 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.364207 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.364317 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.364479 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":1,"thread_start_us":293,"threads_started":1}
I20260809 22:45:47.365936 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ec633524c04f0f90e1714aff03f182" candidate_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" is_pre_election: true
I20260809 22:45:47.366247 27402 raft_consensus.cc:2454] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0a4f9587a2c4c17890f0b4de235bc76 in term 0.
I20260809 22:45:47.366484 27654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ec633524c04f0f90e1714aff03f182" candidate_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a384347a935475b884c14f971c4e301" is_pre_election: true
I20260809 22:45:47.366884 27654 raft_consensus.cc:2454] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0a4f9587a2c4c17890f0b4de235bc76 in term 0.
I20260809 22:45:47.366863 27773 leader_election.cc:302] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0a4f9587a2c4c17890f0b4de235bc76, bc8178dddac146fa9bf1a92bdca2ca99; no voters:
I20260809 22:45:47.367313 28056 raft_consensus.cc:2781] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:47.367434 28056 raft_consensus.cc:482] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:47.367571 28056 raft_consensus.cc:3037] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:47.449901 28056 raft_consensus.cc:504] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.450810 28056 leader_election.cc:288] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [CANDIDATE]: Term 1 election: Requested vote from peers bc8178dddac146fa9bf1a92bdca2ca99 (127.24.136.1:39309), 3a384347a935475b884c14f971c4e301 (127.24.136.2:36767)
I20260809 22:45:47.451552 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ec633524c04f0f90e1714aff03f182" candidate_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8178dddac146fa9bf1a92bdca2ca99"
I20260809 22:45:47.451643 27654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52ec633524c04f0f90e1714aff03f182" candidate_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a384347a935475b884c14f971c4e301"
I20260809 22:45:47.451867 27654 raft_consensus.cc:3037] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:47.451858 27402 raft_consensus.cc:3037] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:47.495129 28061 consensus_queue.cc:1035] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:47.502413 28125 consensus_queue.cc:1035] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:47.549954 27402 raft_consensus.cc:2454] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0a4f9587a2c4c17890f0b4de235bc76 in term 1.
I20260809 22:45:47.549974 27654 raft_consensus.cc:2454] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0a4f9587a2c4c17890f0b4de235bc76 in term 1.
I20260809 22:45:47.550789 27776 leader_election.cc:302] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a384347a935475b884c14f971c4e301, a0a4f9587a2c4c17890f0b4de235bc76, bc8178dddac146fa9bf1a92bdca2ca99; no voters:
I20260809 22:45:47.551326 28056 raft_consensus.cc:2781] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:47.552023 28056 raft_consensus.cc:686] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 LEADER]: Becoming Leader. State: Replica: a0a4f9587a2c4c17890f0b4de235bc76, State: Running, Role: LEADER
I20260809 22:45:47.552616 28056 consensus_queue.cc:237] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } }
I20260809 22:45:47.555493 28137 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: TxnStatusTablet state changed. Reason: New leader a0a4f9587a2c4c17890f0b4de235bc76. Latest consensus state: current_term: 1 leader_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.555961 28137 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:45:47.556957 28140 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:45:47.557271 28140 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:45:47.557739 27120 catalog_manager.cc:5495] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0a4f9587a2c4c17890f0b4de235bc76 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } health_report { overall_health: HEALTHY } } }
I20260809 22:45:47.666198 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.646134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47504 (local address 127.24.136.3:40117)
0809 22:45:47.657555 (+ 11421us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.657562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.657592 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:45:47.657719 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.657725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.657838 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.658034 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:47.658729 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.660101 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.662376 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.663117 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.663260 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.663959 (+ 699us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:47.664863 (+ 904us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:47.665033 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.665647 (+ 614us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.665754 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11261,"thread_start_us":10860,"threads_started":1}
I20260809 22:45:47.666517 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.645612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:40117 (local address 127.0.0.1:47504)
0809 22:45:47.656635 (+ 11023us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.656669 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.656966 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.658091 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.658097 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.658113 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:45:47.658460 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.658477 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.660297 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.660305 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.662142 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.662165 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.663330 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.663337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.663579 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.663681 (+ 102us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:45:47.665428 (+ 1747us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:45:47.665466 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.665583 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.665771 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10769,"thread_start_us":11542,"threads_started":1}
I20260809 22:45:47.672920 27402 raft_consensus.cc:1261] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99 [term 1 FOLLOWER]: Refusing update from remote peer a0a4f9587a2c4c17890f0b4de235bc76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:47.673135 27654 raft_consensus.cc:1261] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301 [term 1 FOLLOWER]: Refusing update from remote peer a0a4f9587a2c4c17890f0b4de235bc76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:47.674003 28137 consensus_queue.cc:1035] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:47.674489 28056 consensus_queue.cc:1035] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:47.680647 28048 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: TxnStatusTablet state changed. Reason: New leader a0a4f9587a2c4c17890f0b4de235bc76. Latest consensus state: current_term: 1 leader_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.681034 28048 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:47.685420 28124 mvcc.cc:205] Tried to move back new op lower bound from 7316748483258376192 to 7316748482780291072. Current Snapshot: MvccSnapshot[applied={T|T < 7316748483258376192}]
I20260809 22:45:47.686285 28137 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.686594 28137 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:45:47.688645 28048 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.688961 28048 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P bc8178dddac146fa9bf1a92bdca2ca99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:47.699095 28125 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: TxnStatusTablet state changed. Reason: New leader a0a4f9587a2c4c17890f0b4de235bc76. Latest consensus state: current_term: 1 leader_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.699488 28125 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:47.703346 28056 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.703697 28056 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P a0a4f9587a2c4c17890f0b4de235bc76: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:45:47.711531 28125 tablet_replica.cc:439] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8178dddac146fa9bf1a92bdca2ca99" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39309 } } peers { permanent_uuid: "3a384347a935475b884c14f971c4e301" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 36767 } } peers { permanent_uuid: "a0a4f9587a2c4c17890f0b4de235bc76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40117 } } }
I20260809 22:45:47.711890 28125 tablet_replica.cc:442] T 52ec633524c04f0f90e1714aff03f182 P 3a384347a935475b884c14f971c4e301: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:47.735018 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.720285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:36547 (local address 127.24.136.3:40117)
0809 22:45:47.720476 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.720482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.729397 (+ 8915us) server_negotiation.cc:407] Connection header received
0809 22:45:47.729634 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.729638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.729719 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.729804 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:47.730450 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.731259 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.732966 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.733463 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.733596 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.734227 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:47.734349 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:47.734429 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.734606 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.734721 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:45:47.735118 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.720285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:40117 (local address 127.24.136.2:36547)
0809 22:45:47.729224 (+ 8939us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.729244 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.729467 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.729867 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.729872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.729886 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:45:47.730246 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.730257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.731421 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.731425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.732788 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.732805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.733755 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.733760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.733941 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.734016 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:45:47.734470 (+ 454us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:45:47.734482 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.734555 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.734732 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8689,"thread_start_us":8425,"threads_started":1}
I20260809 22:45:47.801705 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.790585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:37369 (local address 127.24.136.2:36767)
0809 22:45:47.790822 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.790829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.792719 (+ 1890us) server_negotiation.cc:407] Connection header received
0809 22:45:47.792949 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.792955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.793055 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.793165 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:47.794270 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.795649 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.798193 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.799779 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.799922 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.800802 (+ 880us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:47.800955 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:47.801024 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.801262 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.801373 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:45:47.802647 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.789879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:36767 (local address 127.24.136.3:37369)
0809 22:45:47.792565 (+ 2686us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.792590 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.792773 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.793199 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.793217 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.793230 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:45:47.794039 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.794057 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.795900 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.795907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.797993 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.798018 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.800100 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.800108 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.800373 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.800513 (+ 140us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:45:47.801110 (+ 597us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:45:47.801129 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.802041 (+ 912us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.802209 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2432,"thread_start_us":2204,"threads_started":1}
I20260809 22:45:47.858495 27118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:47.858798 27118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:47.863040 27118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d54a89bad376427f954adb21af663c39: Sending DeleteTablet for 3 replicas of tablet d73454de97b849ff916534973ccacfdd
I20260809 22:45:47.864667 27606 tablet_service.cc:1514] Processing DeleteTablet for tablet d73454de97b849ff916534973ccacfdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:47 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:48952
I20260809 22:45:47.864691 27874 tablet_service.cc:1514] Processing DeleteTablet for tablet d73454de97b849ff916534973ccacfdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:47 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:47488
I20260809 22:45:47.865605 27363 tablet_service.cc:1514] Processing DeleteTablet for tablet d73454de97b849ff916534973ccacfdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:47 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:48334
I20260809 22:45:47.879164 28193 tablet_replica.cc:330] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: stopping tablet replica
I20260809 22:45:47.879519 28193 raft_consensus.cc:2229] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:47.879873 28193 raft_consensus.cc:2258] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:47.881335 28193 ts_tablet_manager.cc:1905] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:47.883776 28188 tablet_replica.cc:330] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: stopping tablet replica
I20260809 22:45:47.884024 28188 raft_consensus.cc:2229] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:45:47.884476 28196 tablet_replica.cc:330] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: stopping tablet replica
I20260809 22:45:47.884708 28196 raft_consensus.cc:2229] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:47.884727 28188 raft_consensus.cc:2258] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:47.885027 28196 raft_consensus.cc:2258] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:47.886039 28188 ts_tablet_manager.cc:1905] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:47.886322 28196 ts_tablet_manager.cc:1905] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 22:45:47 dist-test-slave-pww0 krb5kdc[27032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:45:47.911717 28199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.880989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.24.136.1:39309)
0809 22:45:47.883327 (+ 2338us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.883334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.895383 (+ 12049us) server_negotiation.cc:407] Connection header received
0809 22:45:47.895609 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.895616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.895729 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.895874 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:47.897516 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.898336 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.899781 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.900286 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.900371 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.905061 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:47.905079 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:47.905096 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:47.905146 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:47.908471 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.909310 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.909319 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.909321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.909483 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.909905 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.909909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.909911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.910330 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:47.910652 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.911017 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.911267 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2181,"thread_start_us":2075,"threads_started":1}
W20260809 22:45:47.912559 27486 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48342
I20260809 22:45:47.943665 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.915890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.24.136.62:32925)
0809 22:45:47.929717 (+ 13827us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.929725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.930340 (+ 615us) server_negotiation.cc:407] Connection header received
0809 22:45:47.930565 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.930571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.930664 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.930772 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:47.932322 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.933155 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.934520 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.934973 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.935097 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.937720 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:47.937726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:47.937730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:47.937768 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:47.940587 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.941418 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.941422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.941425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.941557 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.941963 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.941966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.941968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.942274 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:47.942509 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.942887 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.943138 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13636,"thread_start_us":13326,"threads_started":1}
W20260809 22:45:47.944383 27118 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42552
I20260809 22:45:47.946224 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27247
I20260809 22:45:47.961910 28196 ts_tablet_manager.cc:1918] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:45:47.962085 28196 log.cc:1196] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/d73454de97b849ff916534973ccacfdd
I20260809 22:45:47.962559 28196 ts_tablet_manager.cc:1939] T d73454de97b849ff916534973ccacfdd P bc8178dddac146fa9bf1a92bdca2ca99: Deleting consensus metadata
I20260809 22:45:47.966149 28199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.947281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48354 (local address 127.24.136.1:39309)
0809 22:45:47.948730 (+ 1449us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.948736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.949265 (+ 529us) server_negotiation.cc:407] Connection header received
0809 22:45:47.949564 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.949571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.949652 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.949735 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:47.951369 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.952449 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.953915 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.954507 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.954635 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.957328 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:47.957333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:47.957337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:47.957378 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:47.961531 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.963145 (+ 1614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.963152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.963156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.963363 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.964016 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.964022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.964025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.964585 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:47.964872 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.965525 (+ 653us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.965815 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 22:45:47.967947 27486 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48354
W20260809 22:45:47.968544 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27247: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:45:47.968756 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 27247
W20260809 22:45:48.046653 27086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.136.1:39309 (error 108)
I20260809 22:45:48.047046 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27504
I20260809 22:45:48.049522 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:48.048328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39309 (local address 127.0.0.1:48364)
0809 22:45:48.048935 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:48.049140 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":303,"threads_started":1}
W20260809 22:45:48.049822 27086 catalog_manager.cc:4584] TS bc8178dddac146fa9bf1a92bdca2ca99 (127.24.136.1:39309): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d73454de97b849ff916534973ccacfdd: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39309: connect: Connection refused (error 111)
Aug 09 22:45:48 dist-test-slave-pww0 krb5kdc[27032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:45:48.061188 28193 ts_tablet_manager.cc:1918] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:45:48.061385 28193 log.cc:1196] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/d73454de97b849ff916534973ccacfdd
I20260809 22:45:48.061561 28188 ts_tablet_manager.cc:1918] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:45:48.061720 28188 log.cc:1196] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/d73454de97b849ff916534973ccacfdd
I20260809 22:45:48.061946 28193 ts_tablet_manager.cc:1939] T d73454de97b849ff916534973ccacfdd P a0a4f9587a2c4c17890f0b4de235bc76: Deleting consensus metadata
I20260809 22:45:48.062225 28188 ts_tablet_manager.cc:1939] T d73454de97b849ff916534973ccacfdd P 3a384347a935475b884c14f971c4e301: Deleting consensus metadata
I20260809 22:45:48.066032 27091 catalog_manager.cc:4841] TS a0a4f9587a2c4c17890f0b4de235bc76 (127.24.136.3:40117): tablet d73454de97b849ff916534973ccacfdd (table test-table [id=2841ad565b594349a3f4b6466e4c0594]) successfully deleted
I20260809 22:45:48.069191 27091 catalog_manager.cc:4841] TS 3a384347a935475b884c14f971c4e301 (127.24.136.2:36767): tablet d73454de97b849ff916534973ccacfdd (table test-table [id=2841ad565b594349a3f4b6466e4c0594]) successfully deleted
I20260809 22:45:48.069366 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:48.048024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.24.136.2:36767)
0809 22:45:48.048287 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:48.048294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:48.048383 (+ 89us) server_negotiation.cc:407] Connection header received
0809 22:45:48.048626 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:48.048633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:48.048732 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:48.048811 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:48.050453 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:48.051675 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:48.053367 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:48.054059 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:48.054212 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:48.059061 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:48.059087 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:48.059110 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:48.059168 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:48.064861 (+ 5693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:48.066456 (+ 1595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:48.066469 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:48.066473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:48.066699 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:48.067294 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:48.067299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:48.067301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:48.067871 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:48.068183 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:48.068590 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 22:45:48.068845 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260809 22:45:48.070133 27736 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48968
W20260809 22:45:48.070963 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27504: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:45:48.071166 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 27504
I20260809 22:45:48.104007 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:48.103378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39309 (local address 127.0.0.1:48372)
0809 22:45:48.103677 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:48.103813 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260809 22:45:48.105352 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:48.104824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39309 (local address 127.0.0.1:48386)
0809 22:45:48.105024 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:48.105139 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 22:45:48.113054 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27748
Aug 09 22:45:48 dist-test-slave-pww0 krb5kdc[27032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:45:48.131845 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:48.113978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.24.136.3:40117)
0809 22:45:48.114183 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:48.114188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:48.114209 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:45:48.114385 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:48.114389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:48.114481 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:48.114582 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:48.116179 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:48.116984 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:48.118404 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:48.118935 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:48.119065 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:48.124293 (+ 5228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:48.124314 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:48.124332 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:48.124383 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:48.127381 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:48.128482 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:48.128495 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:48.128498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:48.128722 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:48.129273 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:48.129279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:48.129282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:48.129824 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:48.130095 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:48.130520 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 22:45:48.130762 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 22:45:48.132761 27986 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47514
W20260809 22:45:48.133600 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27748: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:45:48.133765 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 27748
I20260809 22:45:48.166464 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:48.165876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39309 (local address 127.0.0.1:48388)
0809 22:45:48.166145 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:48.166268 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 22:45:48.167845 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:48.167364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39309 (local address 127.0.0.1:48392)
0809 22:45:48.167580 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:48.167681 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 22:45:48.170657 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27056
I20260809 22:45:48.171814 27195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:48.293708 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:48.293107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39309 (local address 127.0.0.1:48394)
0809 22:45:48.293371 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:48.293485 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 22:45:48.294190 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 27056
2026-08-09T22:45:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5815 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:45:48 dist-test-slave-pww0 krb5kdc[28215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:45:48 dist-test-slave-pww0 krb5kdc[28215](info): set up 2 sockets
Aug 09 22:45:48 dist-test-slave-pww0 krb5kdc[28215](info): commencing operation
krb5kdc: starting...
W20260809 22:45:49.702756 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.358s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:45:49 dist-test-slave-pww0 krb5kdc[28215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:45:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:45:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:45:49.928009 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:34923
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:34923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:50.278298 28244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:50.278689 28244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:50.278838 28244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:50.286496 28244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:45:50.286653 28244 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:45:50.286728 28244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:50.286814 28244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:50.286883 28244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:45:50.286968 28244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:45:50.297480 28244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:34923
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:34923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:50.298942 28244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:50.300405 28244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:50.313002 28250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:50.313030 28251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:50.312947 28254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:50.313853 28244 server_base.cc:1034] running on GCE node
I20260809 22:45:50.314709 28244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:50.316522 28244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:50.317780 28244 hybrid_clock.cc:648] HybridClock initialized: now 1786315550317714 us; error 37 us; skew 500 ppm
I20260809 22:45:50.318154 28244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:50 dist-test-slave-pww0 krb5kdc[28215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315550, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:50.324997 28244 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:45:50.326712 28244 webserver.cc:460] Webserver started at http://127.24.136.62:46559/ using document root <none> and password file <none>
I20260809 22:45:50.327564 28244 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:50.327693 28244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:50.328054 28244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:50.331398 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "8364e77f644a40ec833c0804b7815b78"
format_stamp: "Formatted at 2026-08-09 22:45:50 on dist-test-slave-pww0"
server_key: "640cb179fb8d7e539c56b79687089861"
server_key_iv: "b4b5f90f033b1d464ea6974359d7d355"
server_key_version: "encryptionkey@0"
I20260809 22:45:50.332389 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "8364e77f644a40ec833c0804b7815b78"
format_stamp: "Formatted at 2026-08-09 22:45:50 on dist-test-slave-pww0"
server_key: "640cb179fb8d7e539c56b79687089861"
server_key_iv: "b4b5f90f033b1d464ea6974359d7d355"
server_key_version: "encryptionkey@0"
I20260809 22:45:50.338457 28244 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 22:45:50.342679 28264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:50.344516 28244 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:45:50.344702 28244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "8364e77f644a40ec833c0804b7815b78"
format_stamp: "Formatted at 2026-08-09 22:45:50 on dist-test-slave-pww0"
server_key: "640cb179fb8d7e539c56b79687089861"
server_key_iv: "b4b5f90f033b1d464ea6974359d7d355"
server_key_version: "encryptionkey@0"
I20260809 22:45:50.344923 28244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:50.358562 28244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:50.363281 28244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:50.363606 28244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:50.386756 28244 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:34923
I20260809 22:45:50.386778 28371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:34923 every 8 connection(s)
I20260809 22:45:50.388298 28244 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:45:50.391173 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 28244
I20260809 22:45:50.391389 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:45:50.392094 25120 external_mini_cluster.cc:1428] Setting key 4e269b53d1a75479b67c9dbcad22b24b
I20260809 22:45:50.392664 28373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:50.407768 28373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78: Bootstrap starting.
Aug 09 22:45:50 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:50.411348 28373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:50.412545 28373 log.cc:824] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:50.415850 28373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78: No bootstrap required, opened a new log
I20260809 22:45:50.417407 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:50.394147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41872 (local address 127.24.136.62:34923)
0809 22:45:50.394939 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:50.394951 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:50.394993 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:45:50.397449 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:50.397478 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:50.397977 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:50.398367 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:50.400395 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:50.401799 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:50.403706 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:50.404561 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:50.404739 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:50.409269 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:50.409282 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:50.409298 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:50.409357 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:50.412510 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:50.413305 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:50.413312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:50.413320 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:50.413511 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:50.413930 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:50.413934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:50.413936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:50.414538 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:50.414837 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:50.415379 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 22:45:50.415891 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":371,"threads_started":1}
I20260809 22:45:50.424311 28373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8364e77f644a40ec833c0804b7815b78" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34923 } }
I20260809 22:45:50.424736 28373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:50.424820 28373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8364e77f644a40ec833c0804b7815b78, State: Initialized, Role: FOLLOWER
I20260809 22:45:50.425514 28373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8364e77f644a40ec833c0804b7815b78" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34923 } }
I20260809 22:45:50.425758 28373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:45:50.425904 28373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:45:50.426075 28373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:50.428402 28373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8364e77f644a40ec833c0804b7815b78" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34923 } }
I20260809 22:45:50.428992 28373 leader_election.cc:302] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8364e77f644a40ec833c0804b7815b78; no voters:
I20260809 22:45:50.429608 28373 leader_election.cc:288] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:45:50.429767 28383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:50.430953 28383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [term 1 LEADER]: Becoming Leader. State: Replica: 8364e77f644a40ec833c0804b7815b78, State: Running, Role: LEADER
I20260809 22:45:50.431649 28383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8364e77f644a40ec833c0804b7815b78" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34923 } }
I20260809 22:45:50.431854 28373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:45:50.434911 28388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8364e77f644a40ec833c0804b7815b78. Latest consensus state: current_term: 1 leader_uuid: "8364e77f644a40ec833c0804b7815b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8364e77f644a40ec833c0804b7815b78" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34923 } } }
I20260809 22:45:50.435356 28388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:50.437383 28386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8364e77f644a40ec833c0804b7815b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8364e77f644a40ec833c0804b7815b78" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34923 } } }
I20260809 22:45:50.437983 28386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78 [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:50.438738 28397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:45:50.444319 28397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:45:50.455068 28397 catalog_manager.cc:1347] Generated new cluster ID: 7c6983fd2531402da98f20039698811e
I20260809 22:45:50.455289 28397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:45:50.480635 28397 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:45:50.482085 28397 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:45:50.500044 28397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78: Generated new TSK 0
I20260809 22:45:50.501010 28397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:45:50.559755 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34923
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:50.917800 28418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:50.918164 28418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:50.918316 28418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:50.925917 28418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:50.926081 28418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:50.926214 28418 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:45:50.926287 28418 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:45:50.926396 28418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:45:50.937175 28418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.136.62:34923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:50.938750 28418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:50.940225 28418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:50.955332 28424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:50.955494 28428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:50.955359 28425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:50.955940 28418 server_base.cc:1034] running on GCE node
I20260809 22:45:50.956843 28418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:50.957943 28418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:50.959215 28418 hybrid_clock.cc:648] HybridClock initialized: now 1786315550959145 us; error 57 us; skew 500 ppm
I20260809 22:45:50.959664 28418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:50 dist-test-slave-pww0 krb5kdc[28215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315550, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:50.963497 28418 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:45:50.965253 28418 webserver.cc:460] Webserver started at http://127.24.136.1:34109/ using document root <none> and password file <none>
I20260809 22:45:50.966050 28418 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:50.966194 28418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:50.966554 28418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:50.970117 28418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "f2c194e751e34e388c830dc9c2091d47"
format_stamp: "Formatted at 2026-08-09 22:45:50 on dist-test-slave-pww0"
server_key: "7518afaa98d370d893eb0c4d7eb9c2b5"
server_key_iv: "8e1b451493967faffd371e16342ddb06"
server_key_version: "encryptionkey@0"
I20260809 22:45:50.971129 28418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "f2c194e751e34e388c830dc9c2091d47"
format_stamp: "Formatted at 2026-08-09 22:45:50 on dist-test-slave-pww0"
server_key: "7518afaa98d370d893eb0c4d7eb9c2b5"
server_key_iv: "8e1b451493967faffd371e16342ddb06"
server_key_version: "encryptionkey@0"
I20260809 22:45:50.978029 28418 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 22:45:50.982949 28437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:50.984711 28418 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 22:45:50.984938 28418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "f2c194e751e34e388c830dc9c2091d47"
format_stamp: "Formatted at 2026-08-09 22:45:50 on dist-test-slave-pww0"
server_key: "7518afaa98d370d893eb0c4d7eb9c2b5"
server_key_iv: "8e1b451493967faffd371e16342ddb06"
server_key_version: "encryptionkey@0"
I20260809 22:45:50.985189 28418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:51.009665 28418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:51.014397 28418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:51.014709 28418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:51.017668 28418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:51.017797 28418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:51.017923 28418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:51.018026 28418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:45:51 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315550, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:51.044064 28454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:51.018238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:42243 (local address 127.24.136.62:34923)
0809 22:45:51.018961 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:51.018968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:51.020172 (+ 1204us) server_negotiation.cc:407] Connection header received
0809 22:45:51.022352 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:51.022359 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:51.022462 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:51.022579 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:51.025499 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.026462 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.029005 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.029729 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.029895 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:51.035642 (+ 5747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:51.035651 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:51.035656 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:51.035712 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:51.039598 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:51.040848 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:51.040855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:51.040859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:51.041057 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:51.041626 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:51.041632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:51.041636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:51.042034 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:51.042214 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:51.043364 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0809 22:45:51.043668 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":476,"threads_started":1}
I20260809 22:45:51.045756 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:51.018970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34923 (local address 127.24.136.1:42243)
0809 22:45:51.019969 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:51.020019 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:51.022016 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:51.022950 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:51.022966 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:51.024127 (+ 1161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:51.025222 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:51.025260 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.026671 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.026675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:51.028791 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:51.028807 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.030091 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.030101 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:51.030475 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:51.031377 (+ 902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:51.031422 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:51.035239 (+ 3817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:51.039771 (+ 4532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:51.039783 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:51.039808 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:51.040632 (+ 824us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:51.041220 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:51.041226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:51.041233 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:51.041488 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:51.042202 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:51.042216 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:51.042811 (+ 595us) client_negotiation.cc:769] Sending connection context
0809 22:45:51.043219 (+ 408us) client_negotiation.cc:240] Negotiation successful
0809 22:45:51.043925 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":401,"threads_started":1}
I20260809 22:45:51.057591 28418 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:39705
I20260809 22:45:51.057663 28647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:39705 every 8 connection(s)
I20260809 22:45:51.059315 28418 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:45:51.061704 28648 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34923
I20260809 22:45:51.061944 28648 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:51.062822 28648 heartbeater.cc:511] Master 127.24.136.62:34923 requested a full tablet report, sending...
I20260809 22:45:51.064585 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 28418
I20260809 22:45:51.064810 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:45:51.065593 25120 external_mini_cluster.cc:1428] Setting key 5f328580b2f95af2b9c126675493e89f
I20260809 22:45:51.065577 28286 ts_manager.cc:194] Registered new tserver with Master: f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705)
I20260809 22:45:51.067909 28286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:42243
I20260809 22:45:51.071314 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:51.053948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34923 (local address 127.24.136.1:55579)
0809 22:45:51.056517 (+ 2569us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:51.056540 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:51.056741 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:51.057392 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:51.057398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:51.058147 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:51.058549 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:51.058564 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.059753 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.059759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:51.061302 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:51.061317 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.062199 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.062207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:51.062398 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:51.063147 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:51.063169 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:51.064282 (+ 1113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:51.068071 (+ 3789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:51.068077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:51.068081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:51.068611 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:51.069092 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:51.069105 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:51.069109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:51.069308 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:51.070069 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:51.070075 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:51.070321 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 22:45:51.070490 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 22:45:51.070845 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2412,"thread_start_us":2266,"threads_started":1}
I20260809 22:45:51.071341 28454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:51.053990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:55579 (local address 127.24.136.62:34923)
0809 22:45:51.054212 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:51.054220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:51.056686 (+ 2466us) server_negotiation.cc:407] Connection header received
0809 22:45:51.056890 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:51.056896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:51.057021 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:51.057174 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:51.058732 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.059599 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.061445 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.062004 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.062128 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:51.064514 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:51.064522 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:51.064527 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:51.064577 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:51.067917 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:51.068756 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:51.068762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:51.068766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:51.068950 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:51.069473 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:51.069479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:51.069483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:51.069909 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:51.070092 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:51.070656 (+ 564us) server_negotiation.cc:299] Negotiation successful
0809 22:45:51.070926 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:45:51.119552 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34923
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:51.481528 28652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:51.481935 28652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:51.482095 28652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:51.489670 28652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:51.489799 28652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:51.489902 28652 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:45:51.489984 28652 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:45:51.490062 28652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:45:51.500526 28652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.136.62:34923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:51.501946 28652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:51.503353 28652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:51.518021 28666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:51.518049 28664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:51.518417 28662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:51.518831 28652 server_base.cc:1034] running on GCE node
I20260809 22:45:51.519565 28652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:51.520623 28652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:51.521860 28652 hybrid_clock.cc:648] HybridClock initialized: now 1786315551521798 us; error 54 us; skew 500 ppm
I20260809 22:45:51.522224 28652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:51 dist-test-slave-pww0 krb5kdc[28215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315551, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:51.525801 28652 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:45:51.527621 28652 webserver.cc:460] Webserver started at http://127.24.136.2:35855/ using document root <none> and password file <none>
I20260809 22:45:51.528365 28652 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:51.528498 28652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:51.528827 28652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:51.532228 28652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "291ab60c3be34d47ad0c127139bf1f62"
format_stamp: "Formatted at 2026-08-09 22:45:51 on dist-test-slave-pww0"
server_key: "6020cc084a067faede916fa7429b4de8"
server_key_iv: "a78bbf30e2231fd8297a3241cc398590"
server_key_version: "encryptionkey@0"
I20260809 22:45:51.533115 28652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "291ab60c3be34d47ad0c127139bf1f62"
format_stamp: "Formatted at 2026-08-09 22:45:51 on dist-test-slave-pww0"
server_key: "6020cc084a067faede916fa7429b4de8"
server_key_iv: "a78bbf30e2231fd8297a3241cc398590"
server_key_version: "encryptionkey@0"
I20260809 22:45:51.539245 28652 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 22:45:51.543349 28674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:51.544960 28652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:45:51.545176 28652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "291ab60c3be34d47ad0c127139bf1f62"
format_stamp: "Formatted at 2026-08-09 22:45:51 on dist-test-slave-pww0"
server_key: "6020cc084a067faede916fa7429b4de8"
server_key_iv: "a78bbf30e2231fd8297a3241cc398590"
server_key_version: "encryptionkey@0"
I20260809 22:45:51.545416 28652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:51.560674 28652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:51.565586 28652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:51.565919 28652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:51.568972 28652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:51.569108 28652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:51.569253 28652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:51.569353 28652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:45:51 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315551, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:51.594285 28454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:51.569524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:56015 (local address 127.24.136.62:34923)
0809 22:45:51.569751 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:51.569758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:51.571389 (+ 1631us) server_negotiation.cc:407] Connection header received
0809 22:45:51.573669 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:51.573675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:51.573770 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:51.573865 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:51.576463 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.577678 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.580181 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.580776 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.580933 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:51.586223 (+ 5290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:51.586229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:51.586233 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:51.586273 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:51.589367 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:51.590867 (+ 1500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:51.590874 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:51.590879 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:51.591039 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:51.591736 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:51.591742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:51.591745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:51.592126 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:51.592317 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:51.593583 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0809 22:45:51.593886 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:45:51.596385 28692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:51.570245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34923 (local address 127.24.136.2:56015)
0809 22:45:51.571184 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:51.571234 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:51.573313 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:51.574209 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:51.574222 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:51.575112 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:51.576152 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:51.576185 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.577879 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.577885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:51.579973 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:51.579990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.581066 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.581075 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:51.581483 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:51.582510 (+ 1027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:51.582551 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:51.585930 (+ 3379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:51.589632 (+ 3702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:51.589646 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:51.589674 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:51.590663 (+ 989us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:51.591171 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:51.591184 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:51.591188 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:51.591559 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:51.592288 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:51.592302 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:51.592971 (+ 669us) client_negotiation.cc:769] Sending connection context
0809 22:45:51.593429 (+ 458us) client_negotiation.cc:240] Negotiation successful
0809 22:45:51.594324 (+ 895us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":422,"threads_started":1}
I20260809 22:45:51.617233 28652 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:42377
I20260809 22:45:51.617331 28877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:42377 every 8 connection(s)
I20260809 22:45:51.618944 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:51.602837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34923 (local address 127.24.136.2:46969)
0809 22:45:51.603608 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:51.603632 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:51.603911 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:51.604386 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:51.604395 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:51.604874 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:51.605185 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:51.605202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.606500 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.606505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:51.608210 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:51.608253 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.609112 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.609119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:51.609304 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:51.609960 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:51.609986 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:51.611147 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:51.615632 (+ 4485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:51.615637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:51.615640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:51.616117 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:51.616649 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:51.616656 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:51.616665 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:51.616858 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:51.617692 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:51.617697 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:51.617926 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 22:45:51.618123 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 22:45:51.618496 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":375,"threads_started":1}
I20260809 22:45:51.619016 28652 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:45:51.619976 28454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:51.602854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:46969 (local address 127.24.136.62:34923)
0809 22:45:51.603092 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:51.603100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:51.603763 (+ 663us) server_negotiation.cc:407] Connection header received
0809 22:45:51.604121 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:51.604127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:51.604225 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:51.604321 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:51.605361 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.606336 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.608413 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:51.608891 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:51.609017 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:51.611440 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:51.611448 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:51.611477 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:51.611527 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:51.615412 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:51.616271 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:51.616277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:51.616281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:51.616488 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:51.617000 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:51.617006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:51.617010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:51.617567 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:51.617701 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:51.619327 (+ 1626us) server_negotiation.cc:299] Negotiation successful
0809 22:45:51.619629 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:45:51.621450 28878 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34923
I20260809 22:45:51.621721 28878 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:51.622438 28878 heartbeater.cc:511] Master 127.24.136.62:34923 requested a full tablet report, sending...
I20260809 22:45:51.624127 28285 ts_manager.cc:194] Registered new tserver with Master: 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377)
I20260809 22:45:51.625015 28285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:56015
I20260809 22:45:51.629154 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 28652
I20260809 22:45:51.629343 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:45:51.629796 25120 external_mini_cluster.cc:1428] Setting key 4a0ae622602c5584f4bb458d68b167c2
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:45:51.679160 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34923
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:52.031363 28886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:52.031795 28886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:52.031965 28886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:52.039757 28886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:52.039912 28886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:52.040045 28886 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:45:52.040112 28886 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:45:52.040225 28886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:45:52.051221 28886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.136.62:34923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:52.052743 28886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:52.054162 28886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:52.068365 28895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:52.068336 28904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:52.068295 28909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:52.069892 28886 server_base.cc:1034] running on GCE node
I20260809 22:45:52.070765 28886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:52.071982 28886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:52.072219 28648 heartbeater.cc:503] Master 127.24.136.62:34923 was elected leader, sending a full tablet report...
I20260809 22:45:52.073240 28886 hybrid_clock.cc:648] HybridClock initialized: now 1786315552073153 us; error 70 us; skew 500 ppm
I20260809 22:45:52.073597 28886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:52 dist-test-slave-pww0 krb5kdc[28215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:52.077106 28886 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:45:52.078748 28886 webserver.cc:460] Webserver started at http://127.24.136.3:41331/ using document root <none> and password file <none>
I20260809 22:45:52.079524 28886 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:52.079658 28886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:52.079954 28886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:52.083278 28886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0"
format_stamp: "Formatted at 2026-08-09 22:45:52 on dist-test-slave-pww0"
server_key: "020f0bf18164c2923de537888b9c1d79"
server_key_iv: "f71176708ea2f9fe9e71dad49c3ca8fe"
server_key_version: "encryptionkey@0"
I20260809 22:45:52.084161 28886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0"
format_stamp: "Formatted at 2026-08-09 22:45:52 on dist-test-slave-pww0"
server_key: "020f0bf18164c2923de537888b9c1d79"
server_key_iv: "f71176708ea2f9fe9e71dad49c3ca8fe"
server_key_version: "encryptionkey@0"
I20260809 22:45:52.090232 28886 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 22:45:52.095021 28917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:52.096647 28886 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:45:52.096850 28886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0"
format_stamp: "Formatted at 2026-08-09 22:45:52 on dist-test-slave-pww0"
server_key: "020f0bf18164c2923de537888b9c1d79"
server_key_iv: "f71176708ea2f9fe9e71dad49c3ca8fe"
server_key_version: "encryptionkey@0"
I20260809 22:45:52.097071 28886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:52.128383 28886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:52.134735 28886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:52.135089 28886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:52.138932 28886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:52.139076 28886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:52.139230 28886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:52.139324 28886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:45:52 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:52.176105 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.139431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53021 (local address 127.24.136.62:34923)
0809 22:45:52.152025 (+ 12594us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.152033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.152061 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:45:52.152135 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.152139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.152217 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.152361 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:52.155177 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.156058 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.158807 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.159283 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.159422 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.167492 (+ 8070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:52.167501 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:52.167507 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:52.167564 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:52.170752 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:52.172300 (+ 1548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:52.172309 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:52.172314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:52.172592 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:52.173335 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:52.173341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:52.173344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:52.173774 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:52.173937 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.175181 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.175567 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12442,"thread_start_us":12305,"threads_started":1}
I20260809 22:45:52.177160 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.140312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34923 (local address 127.24.136.3:53021)
0809 22:45:52.141336 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.141399 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.144217 (+ 2818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.152827 (+ 8610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.152846 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.153750 (+ 904us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:52.154909 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.154947 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.156279 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.156284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.158590 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.158615 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.159603 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.159613 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.159939 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.161017 (+ 1078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:52.161057 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:52.164830 (+ 3773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:52.171166 (+ 6336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:52.171179 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:52.171202 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:52.172083 (+ 881us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:52.172773 (+ 690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:52.172779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:52.172783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:52.173123 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:52.174053 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:52.174064 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:52.174653 (+ 589us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.175005 (+ 352us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.175770 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":400,"threads_started":1}
I20260809 22:45:52.198954 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.183412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:40629 (local address 127.24.136.62:34923)
0809 22:45:52.183611 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.183617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.184435 (+ 818us) server_negotiation.cc:407] Connection header received
0809 22:45:52.184605 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.184611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.184684 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.184768 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:52.185764 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.186487 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.188529 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.189069 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.189185 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.191549 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:52.191556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:52.191561 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:52.191608 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:52.195709 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:52.196739 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:52.196744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:52.196746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:52.196918 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:52.197393 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:52.197399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:52.197403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:52.197716 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:52.197878 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.198341 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.198579 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:45:52.199024 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.183633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34923 (local address 127.24.136.3:40629)
0809 22:45:52.184314 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.184330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.184483 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.184821 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.184826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.185286 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:52.185594 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.185614 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.186648 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.186652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.188323 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.188346 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.189487 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.189494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.189800 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.190498 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:52.190516 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:52.191314 (+ 798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:52.195912 (+ 4598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:52.195919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:52.195923 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:52.196535 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:52.197088 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:52.197094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:52.197098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:52.197268 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:52.197849 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:52.197855 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:52.198046 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.198196 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.198569 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":379,"threads_started":1}
I20260809 22:45:52.213822 28886 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:44855
I20260809 22:45:52.213845 29126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:44855 every 8 connection(s)
I20260809 22:45:52.215207 28886 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:45:52.217590 29130 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34923
I20260809 22:45:52.217829 29130 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:52.218535 29130 heartbeater.cc:511] Master 127.24.136.62:34923 requested a full tablet report, sending...
I20260809 22:45:52.220077 28285 ts_manager.cc:194] Registered new tserver with Master: 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855)
I20260809 22:45:52.220888 28285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:53021
I20260809 22:45:52.224027 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 28886
I20260809 22:45:52.224187 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:45:52.224599 25120 external_mini_cluster.cc:1428] Setting key 282521dbab4ee8b817cf1da2a1b63753
I20260809 22:45:52.228061 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:45:52 dist-test-slave-pww0 krb5kdc[28215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:45:52 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:52.284559 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.269077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51602 (local address 127.24.136.62:34923)
0809 22:45:52.269323 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.269330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.269638 (+ 308us) server_negotiation.cc:407] Connection header received
0809 22:45:52.269862 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.269867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.269942 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.270032 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:52.271657 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.272406 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.273715 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.274172 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.274296 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.278614 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:52.278620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:52.278624 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:52.278668 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:52.281704 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:52.282493 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:52.282499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:52.282501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:52.282655 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:52.283058 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:52.283063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:52.283065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:52.283378 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:52.283693 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.283984 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.284227 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 22:45:52.289624 28285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:45:52.296746 28285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:52.321645 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.311240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58500 (local address 127.24.136.2:42377)
0809 22:45:52.312285 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.312294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.313022 (+ 728us) server_negotiation.cc:407] Connection header received
0809 22:45:52.313136 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.313145 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.313411 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.314372 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:52.314569 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.316086 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.318946 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.320419 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.320567 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.320791 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.321157 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.321346 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":1133,"threads_started":1}
I20260809 22:45:52.321779 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.311271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:42377 (local address 127.0.0.1:58500)
0809 22:45:52.312309 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.312351 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.312558 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.313666 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.313674 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.313705 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:52.314158 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.314173 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.316233 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.316239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.318751 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.318767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.320650 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.320657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.320865 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.320923 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.321088 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.321300 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":117,"thread_start_us":874,"threads_started":1}
I20260809 22:45:52.322580 29159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.312135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.24.136.1:39705)
0809 22:45:52.313476 (+ 1341us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.313484 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.313513 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:45:52.313615 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.313622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.313841 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.314042 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:52.314860 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.316329 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.318825 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.320418 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.320570 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.320791 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.322098 (+ 1307us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.322288 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1190,"thread_start_us":1073,"threads_started":1}
I20260809 22:45:52.322916 29158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.311902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.0.0.1:49952)
0809 22:45:52.312769 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.312785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.312888 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.314253 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.314257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.314284 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:52.314678 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.314690 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.316461 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.316465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.318611 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.318634 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.320950 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.320955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.321167 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.321200 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.321288 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.321418 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":121,"thread_start_us":1507,"threads_started":1}
I20260809 22:45:52.325037 29160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.313052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44855 (local address 127.0.0.1:58156)
0809 22:45:52.313990 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.314013 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.314164 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.316751 (+ 2587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.316755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.316782 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:52.317483 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.317496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.319813 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.319817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.321528 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.321546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.324176 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.324183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.324374 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.324411 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.324502 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.324648 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":184,"thread_start_us":703,"threads_started":1}
I20260809 22:45:52.325070 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.313599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58156 (local address 127.24.136.3:44855)
0809 22:45:52.316131 (+ 2532us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.316138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.316169 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:45:52.316266 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.316274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.316540 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.316743 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:52.317732 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.319670 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.322202 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.323894 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.324082 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.324341 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.324577 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.324765 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2385,"thread_start_us":2164,"threads_started":1}
I20260809 22:45:52.326352 28766 tablet_service.cc:1467] Processing CreateTablet for tablet 6c25525a0a8d4ae0bcc1ad80da78c99f (DEFAULT_TABLE table=test-table [id=c42450533b1443b4b19c3623104479c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:52.326964 28516 tablet_service.cc:1467] Processing CreateTablet for tablet 6c25525a0a8d4ae0bcc1ad80da78c99f (DEFAULT_TABLE table=test-table [id=c42450533b1443b4b19c3623104479c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:52.328020 28766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c25525a0a8d4ae0bcc1ad80da78c99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:52.328431 28516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c25525a0a8d4ae0bcc1ad80da78c99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:52.328449 29007 tablet_service.cc:1467] Processing CreateTablet for tablet 6c25525a0a8d4ae0bcc1ad80da78c99f (DEFAULT_TABLE table=test-table [id=c42450533b1443b4b19c3623104479c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:52.329867 29007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c25525a0a8d4ae0bcc1ad80da78c99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:52.344355 29163 tablet_bootstrap.cc:492] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: Bootstrap starting.
I20260809 22:45:52.347626 29163 tablet_bootstrap.cc:654] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:52.348488 29169 tablet_bootstrap.cc:492] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: Bootstrap starting.
I20260809 22:45:52.349356 29163 log.cc:824] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:52.350122 29166 tablet_bootstrap.cc:492] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: Bootstrap starting.
I20260809 22:45:52.353030 29169 tablet_bootstrap.cc:654] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:52.353588 29163 tablet_bootstrap.cc:492] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: No bootstrap required, opened a new log
I20260809 22:45:52.353896 29163 ts_tablet_manager.cc:1397] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260809 22:45:52.354311 29166 tablet_bootstrap.cc:654] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:52.354334 29169 log.cc:824] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:52.355655 29166 log.cc:824] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:52.357110 29169 tablet_bootstrap.cc:492] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: No bootstrap required, opened a new log
I20260809 22:45:52.357445 29169 ts_tablet_manager.cc:1397] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:45:52.360145 29166 tablet_bootstrap.cc:492] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: No bootstrap required, opened a new log
I20260809 22:45:52.360460 29166 ts_tablet_manager.cc:1397] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260809 22:45:52.364827 29163 raft_consensus.cc:348] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:45:52.365417 29163 raft_consensus.cc:374] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:52.365526 29163 raft_consensus.cc:729] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dc84bf63e54c8c991c6c9ea5bcfff0, State: Initialized, Role: FOLLOWER
I20260809 22:45:52.365790 29169 raft_consensus.cc:348] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:45:52.366255 29169 raft_consensus.cc:374] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:52.366364 29169 raft_consensus.cc:729] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291ab60c3be34d47ad0c127139bf1f62, State: Initialized, Role: FOLLOWER
I20260809 22:45:52.366281 29163 consensus_queue.cc:260] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:45:52.367019 29169 consensus_queue.cc:260] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:45:52.368134 28878 heartbeater.cc:503] Master 127.24.136.62:34923 was elected leader, sending a full tablet report...
I20260809 22:45:52.368901 29169 ts_tablet_manager.cc:1428] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 22:45:52.370069 29130 heartbeater.cc:503] Master 127.24.136.62:34923 was elected leader, sending a full tablet report...
I20260809 22:45:52.370747 29163 ts_tablet_manager.cc:1428] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent starting tablet: real 0.017s user 0.011s sys 0.007s
W20260809 22:45:52.371665 28879 tablet.cc:2369] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:45:52.371857 29166 raft_consensus.cc:348] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:45:52.372772 29166 raft_consensus.cc:374] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:52.372891 29166 raft_consensus.cc:729] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c194e751e34e388c830dc9c2091d47, State: Initialized, Role: FOLLOWER
I20260809 22:45:52.373521 29166 consensus_queue.cc:260] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:45:52.375118 29166 ts_tablet_manager.cc:1428] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260809 22:45:52.467059 29132 tablet.cc:2369] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:45:52.562673 28649 tablet.cc:2369] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:45:52.625650 29177 raft_consensus.cc:482] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:52.625993 29177 raft_consensus.cc:504] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:45:52.628096 29177 leader_election.cc:288] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377), 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855)
I20260809 22:45:52.635004 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.628615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:39085 (local address 127.24.136.2:42377)
0809 22:45:52.628838 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.628843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.629446 (+ 603us) server_negotiation.cc:407] Connection header received
0809 22:45:52.629535 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.629542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.629616 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.629720 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:52.630442 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.631194 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.633165 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.634045 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.634222 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.634310 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.634593 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.634740 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:45:52.635115 29178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.628475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:42377 (local address 127.24.136.1:39085)
0809 22:45:52.629223 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.629243 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.629395 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.629921 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.629924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.629947 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:52.630271 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.630299 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.631347 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.631351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.632992 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.633006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.634275 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.634282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.634427 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.634452 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.634543 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.634693 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":42,"thread_start_us":370,"threads_started":1}
I20260809 22:45:52.635752 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.629632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:51581 (local address 127.24.136.3:44855)
0809 22:45:52.629827 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.629833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.630126 (+ 293us) server_negotiation.cc:407] Connection header received
0809 22:45:52.630334 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.630338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.630401 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.630479 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:52.631030 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.631782 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.633650 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.634632 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.634923 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.635004 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.635380 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.635564 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:45:52.635883 29179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.629397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44855 (local address 127.24.136.1:51581)
0809 22:45:52.629997 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.630016 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.630170 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.630517 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.630521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.630546 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:52.630861 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.630870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.631932 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.631935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.633470 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.633484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.634976 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.634983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.635192 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.635232 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.635333 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.635504 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":1,"thread_start_us":310,"threads_started":1}
I20260809 22:45:52.636475 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c25525a0a8d4ae0bcc1ad80da78c99f" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62" is_pre_election: true
I20260809 22:45:52.636952 29045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c25525a0a8d4ae0bcc1ad80da78c99f" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" is_pre_election: true
I20260809 22:45:52.637163 28798 raft_consensus.cc:2454] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 0.
I20260809 22:45:52.637490 29045 raft_consensus.cc:2454] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 0.
I20260809 22:45:52.638201 28440 leader_election.cc:302] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52dc84bf63e54c8c991c6c9ea5bcfff0, f2c194e751e34e388c830dc9c2091d47; no voters:
I20260809 22:45:52.638633 29177 raft_consensus.cc:2781] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:52.638803 29177 raft_consensus.cc:482] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:52.638916 29177 raft_consensus.cc:3037] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:52.641302 29177 raft_consensus.cc:504] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:45:52.642019 29177 leader_election.cc:288] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 election: Requested vote from peers 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377), 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855)
I20260809 22:45:52.642673 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c25525a0a8d4ae0bcc1ad80da78c99f" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62"
I20260809 22:45:52.642719 29045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c25525a0a8d4ae0bcc1ad80da78c99f" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0"
I20260809 22:45:52.642906 28798 raft_consensus.cc:3037] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:52.642937 29045 raft_consensus.cc:3037] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:52.645385 28798 raft_consensus.cc:2454] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 1.
I20260809 22:45:52.645659 29045 raft_consensus.cc:2454] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 1.
I20260809 22:45:52.646008 28440 leader_election.cc:302] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291ab60c3be34d47ad0c127139bf1f62, f2c194e751e34e388c830dc9c2091d47; no voters:
I20260809 22:45:52.646426 29177 raft_consensus.cc:2781] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:52.646909 29177 raft_consensus.cc:686] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 1 LEADER]: Becoming Leader. State: Replica: f2c194e751e34e388c830dc9c2091d47, State: Running, Role: LEADER
I20260809 22:45:52.647431 29177 consensus_queue.cc:237] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:45:52.652609 28285 catalog_manager.cc:5495] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2c194e751e34e388c830dc9c2091d47 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "f2c194e751e34e388c830dc9c2091d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } health_report { overall_health: HEALTHY } } }
Aug 09 22:45:52 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315550, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:52.724792 29189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.710368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34923 (local address 127.0.0.1:51618)
0809 22:45:52.711064 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.711080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.711249 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.711586 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.711590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.712410 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:52.712797 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.712810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.713871 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.713875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.714851 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.714864 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.715650 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.715656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.715801 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.716476 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:52.716495 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:52.718950 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:52.722067 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:52.722093 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:52.722103 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:52.722556 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:52.722937 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:52.722941 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:52.722943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:52.723084 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:52.723681 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:52.723690 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:52.723846 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.724002 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.724302 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":404,"threads_started":1}
I20260809 22:45:52.724797 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.710351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.24.136.62:34923)
0809 22:45:52.710566 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.710571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.711186 (+ 615us) server_negotiation.cc:407] Connection header received
0809 22:45:52.711371 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.711375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.711435 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.711559 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:52.712950 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.713728 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.714995 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.715421 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.715575 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.719196 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:52.719201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:52.719204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:52.719242 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:52.721900 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:52.722686 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:52.722692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:52.722694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:52.722816 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:52.723193 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:52.723198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:52.723200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:52.723533 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:52.723672 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.724110 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.724462 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":77}
I20260809 22:45:52.730834 28285 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:51618:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:45:52.731989 28285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:52.739588 28516 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe7bd3ad78c4e7985939d92041afec9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3e44f6c6e40a4e27894f570da542b0e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:45:52.739763 28766 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe7bd3ad78c4e7985939d92041afec9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3e44f6c6e40a4e27894f570da542b0e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:45:52.740540 28516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe7bd3ad78c4e7985939d92041afec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:52.740648 28766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe7bd3ad78c4e7985939d92041afec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:52.744242 29007 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe7bd3ad78c4e7985939d92041afec9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3e44f6c6e40a4e27894f570da542b0e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:45:52.745344 29007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe7bd3ad78c4e7985939d92041afec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:52.746279 29166 tablet_bootstrap.cc:492] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: Bootstrap starting.
I20260809 22:45:52.747913 29169 tablet_bootstrap.cc:492] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: Bootstrap starting.
I20260809 22:45:52.748019 29166 tablet_bootstrap.cc:654] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:52.749720 29169 tablet_bootstrap.cc:654] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:52.751150 29166 tablet_bootstrap.cc:492] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: No bootstrap required, opened a new log
I20260809 22:45:52.751314 29166 ts_tablet_manager.cc:1397] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 22:45:52.752108 29166 raft_consensus.cc:348] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:52.752372 29166 raft_consensus.cc:374] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:52.752434 29169 tablet_bootstrap.cc:492] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: No bootstrap required, opened a new log
I20260809 22:45:52.752475 29166 raft_consensus.cc:729] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c194e751e34e388c830dc9c2091d47, State: Initialized, Role: FOLLOWER
I20260809 22:45:52.752570 29169 ts_tablet_manager.cc:1397] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 22:45:52.752710 29166 consensus_queue.cc:260] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:52.753067 29163 tablet_bootstrap.cc:492] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Bootstrap starting.
I20260809 22:45:52.753234 29166 ts_tablet_manager.cc:1428] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 22:45:52.753343 29169 raft_consensus.cc:348] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:52.753590 29169 raft_consensus.cc:374] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:52.753680 29169 raft_consensus.cc:729] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291ab60c3be34d47ad0c127139bf1f62, State: Initialized, Role: FOLLOWER
I20260809 22:45:52.753922 29169 consensus_queue.cc:260] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:52.754372 29175 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.754745 29175 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:52.754846 29163 tablet_bootstrap.cc:654] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:52.755244 29169 ts_tablet_manager.cc:1428] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:45:52.756294 29177 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.756647 29177 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:52.765275 29163 tablet_bootstrap.cc:492] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: No bootstrap required, opened a new log
I20260809 22:45:52.765446 29163 ts_tablet_manager.cc:1397] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20260809 22:45:52.766155 29163 raft_consensus.cc:348] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:52.766363 29163 raft_consensus.cc:374] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:52.766455 29163 raft_consensus.cc:729] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dc84bf63e54c8c991c6c9ea5bcfff0, State: Initialized, Role: FOLLOWER
I20260809 22:45:52.766669 29163 consensus_queue.cc:260] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:52.767108 29174 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.767310 29163 ts_tablet_manager.cc:1428] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:45:52.767434 29174 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:52.798193 29174 raft_consensus.cc:482] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:52.798449 29174 raft_consensus.cc:504] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:52.800194 29174 leader_election.cc:288] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705), 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377)
I20260809 22:45:52.809015 29159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.800780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:39975 (local address 127.24.136.1:39705)
0809 22:45:52.801008 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.801014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.801591 (+ 577us) server_negotiation.cc:407] Connection header received
0809 22:45:52.801749 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.801755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.801835 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.801923 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:52.802560 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.803619 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.806531 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.807783 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.807958 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.808036 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.808638 (+ 602us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.808792 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 22:45:52.809039 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.800595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:51003 (local address 127.24.136.2:42377)
0809 22:45:52.800791 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.800797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.801412 (+ 615us) server_negotiation.cc:407] Connection header received
0809 22:45:52.801617 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.801623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.801708 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.801810 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:52.802365 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.803540 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.806536 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.807783 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.807955 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.808036 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.808591 (+ 555us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.808732 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:45:52.809303 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.800546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:42377 (local address 127.24.136.3:51003)
0809 22:45:52.801273 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.801295 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.801467 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.801832 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.801836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.801856 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:52.802193 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.802206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.803703 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.803709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.806317 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.806339 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.808021 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.808038 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.808386 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.808430 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.808559 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.808753 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":287,"spinlock_wait_cycles":24832,"thread_start_us":629,"threads_started":1}
I20260809 22:45:52.809585 29202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.800546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.24.136.3:39975)
0809 22:45:52.801489 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.801504 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.801615 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.801953 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.801958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.801977 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:52.802411 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.802421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.803741 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.803745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.806317 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.806339 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.808059 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.808065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.808380 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.808435 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.808596 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.808753 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":241,"thread_start_us":340,"threads_started":1}
I20260809 22:45:52.810138 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe7bd3ad78c4e7985939d92041afec9" candidate_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62" is_pre_election: true
I20260809 22:45:52.810496 28798 raft_consensus.cc:2454] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52dc84bf63e54c8c991c6c9ea5bcfff0 in term 0.
I20260809 22:45:52.810711 28555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe7bd3ad78c4e7985939d92041afec9" candidate_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c194e751e34e388c830dc9c2091d47" is_pre_election: true
I20260809 22:45:52.811007 28919 leader_election.cc:302] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291ab60c3be34d47ad0c127139bf1f62, 52dc84bf63e54c8c991c6c9ea5bcfff0; no voters:
I20260809 22:45:52.811235 28555 raft_consensus.cc:2454] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52dc84bf63e54c8c991c6c9ea5bcfff0 in term 0.
I20260809 22:45:52.811527 29174 raft_consensus.cc:2781] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:52.811638 29174 raft_consensus.cc:482] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:52.811712 29174 raft_consensus.cc:3037] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:52.814147 29174 raft_consensus.cc:504] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:52.814797 29174 leader_election.cc:288] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [CANDIDATE]: Term 1 election: Requested vote from peers f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705), 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377)
I20260809 22:45:52.815388 28555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe7bd3ad78c4e7985939d92041afec9" candidate_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c194e751e34e388c830dc9c2091d47"
I20260809 22:45:52.815424 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe7bd3ad78c4e7985939d92041afec9" candidate_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62"
I20260809 22:45:52.815651 28555 raft_consensus.cc:3037] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:52.815670 28798 raft_consensus.cc:3037] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:52.818712 28555 raft_consensus.cc:2454] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52dc84bf63e54c8c991c6c9ea5bcfff0 in term 1.
I20260809 22:45:52.818708 28798 raft_consensus.cc:2454] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52dc84bf63e54c8c991c6c9ea5bcfff0 in term 1.
I20260809 22:45:52.819198 28925 leader_election.cc:302] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52dc84bf63e54c8c991c6c9ea5bcfff0, f2c194e751e34e388c830dc9c2091d47; no voters:
I20260809 22:45:52.819585 29174 raft_consensus.cc:2781] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:52.820093 29174 raft_consensus.cc:686] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 LEADER]: Becoming Leader. State: Replica: 52dc84bf63e54c8c991c6c9ea5bcfff0, State: Running, Role: LEADER
I20260809 22:45:52.820619 29174 consensus_queue.cc:237] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:52.824230 29206 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: TxnStatusTablet state changed. Reason: New leader 52dc84bf63e54c8c991c6c9ea5bcfff0. Latest consensus state: current_term: 1 leader_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.824666 29206 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:45:52.825445 29208 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:45:52.825752 29208 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:45:52.826074 28286 catalog_manager.cc:5495] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } health_report { overall_health: HEALTHY } } }
I20260809 22:45:52.845914 29189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.839230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44855 (local address 127.0.0.1:58162)
0809 22:45:52.839540 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.839563 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.839694 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.840277 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.840281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.840292 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:45:52.840494 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.840501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.842013 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.842018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.843049 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.843063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.844097 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.844101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.844265 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.844333 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:45:52.845414 (+ 1081us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:45:52.845433 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.845515 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.845648 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 22:45:52.845999 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.839367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58162 (local address 127.24.136.3:44855)
0809 22:45:52.839616 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.839622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.839647 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:45:52.839921 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.839927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.840061 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.840245 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:52.840639 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.841854 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.843207 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.843927 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.844073 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.844534 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:52.845303 (+ 769us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:52.845467 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.845573 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.845668 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260809 22:45:52.854840 28555 raft_consensus.cc:1261] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Refusing update from remote peer 52dc84bf63e54c8c991c6c9ea5bcfff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:52.854910 28798 raft_consensus.cc:1261] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Refusing update from remote peer 52dc84bf63e54c8c991c6c9ea5bcfff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:52.855969 29174 consensus_queue.cc:1035] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:52.856422 29206 consensus_queue.cc:1035] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:52.872386 29175 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: TxnStatusTablet state changed. Reason: New leader 52dc84bf63e54c8c991c6c9ea5bcfff0. Latest consensus state: current_term: 1 leader_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.872735 29175 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:52.874953 29177 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: TxnStatusTablet state changed. Reason: New leader 52dc84bf63e54c8c991c6c9ea5bcfff0. Latest consensus state: current_term: 1 leader_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.875324 29177 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:52.878808 29213 mvcc.cc:205] Tried to move back new op lower bound from 7316748504483209216 to 7316748504357187584. Current Snapshot: MvccSnapshot[applied={T|T < 7316748504483209216}]
I20260809 22:45:52.880007 29175 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.880272 29175 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P 291ab60c3be34d47ad0c127139bf1f62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:52.885699 29177 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.886047 29177 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P f2c194e751e34e388c830dc9c2091d47: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:45:52.894482 29206 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.894971 29206 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:45:52.894521 29174 tablet_replica.cc:439] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } } }
I20260809 22:45:52.895722 29174 tablet_replica.cc:442] T 8fe7bd3ad78c4e7985939d92041afec9 P 52dc84bf63e54c8c991c6c9ea5bcfff0: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:45:52.909776 29159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.902426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.24.136.1:39705)
0809 22:45:52.902637 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.902643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.903230 (+ 587us) server_negotiation.cc:407] Connection header received
0809 22:45:52.903430 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.903436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.903607 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.903820 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:52.904321 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.905489 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.906982 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.907703 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.907851 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.908273 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:52.909060 (+ 787us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:52.909210 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.909333 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.909428 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:45:52.929384 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.922514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:58669 (local address 127.24.136.3:44855)
0809 22:45:52.922765 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.922772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.923345 (+ 573us) server_negotiation.cc:407] Connection header received
0809 22:45:52.923671 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.923677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.923791 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.923885 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:52.924417 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.925590 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.927052 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.927798 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.927929 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.928534 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:52.928691 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:52.928795 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.928978 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.929073 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:45:52.929427 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.922467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44855 (local address 127.24.136.1:58669)
0809 22:45:52.923215 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.923237 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.923419 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.923929 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.923933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.923945 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:45:52.924253 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.924262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.925752 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.925757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.926891 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.926905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.928066 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.928078 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.928221 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.928305 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:45:52.928830 (+ 525us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:45:52.928843 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.928927 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.929062 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":392,"threads_started":1}
I20260809 22:45:52.940577 28798 raft_consensus.cc:1261] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Refusing update from remote peer f2c194e751e34e388c830dc9c2091d47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:52.940763 29045 raft_consensus.cc:1261] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Refusing update from remote peer f2c194e751e34e388c830dc9c2091d47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:52.941443 29181 consensus_queue.cc:1035] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:52.941870 29177 consensus_queue.cc:1035] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:52.949134 29214 mvcc.cc:205] Tried to move back new op lower bound from 7316748504836554752 to 7316748503648436224. Current Snapshot: MvccSnapshot[applied={T|T < 7316748504836554752}]
I20260809 22:45:52.951815 29212 mvcc.cc:205] Tried to move back new op lower bound from 7316748504836554752 to 7316748503648436224. Current Snapshot: MvccSnapshot[applied={T|T < 7316748504836554752}]
I20260809 22:45:52.984465 29159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.977535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:41215 (local address 127.24.136.1:39705)
0809 22:45:52.977758 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:52.977765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:52.978436 (+ 671us) server_negotiation.cc:407] Connection header received
0809 22:45:52.978725 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:52.978730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:52.978815 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:52.978903 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:52.979683 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.980510 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.982239 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.982896 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.983045 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.983682 (+ 637us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:52.983809 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:52.983893 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:52.984046 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 22:45:52.984153 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:45:52.984632 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:52.977535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.24.136.3:41215)
0809 22:45:52.978329 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:52.978351 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:52.978508 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:52.978948 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:52.978953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:52.978966 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:45:52.979441 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.979481 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.980805 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.980808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.982073 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:52.982088 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:52.983139 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:52.983147 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:52.983346 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:52.983439 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:45:52.983918 (+ 479us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:45:52.983931 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 22:45:52.984010 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 22:45:52.984173 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":378,"threads_started":1}
I20260809 22:45:53.037032 28285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:53.037365 28285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:53.042182 28285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78: Sending DeleteTablet for 3 replicas of tablet 6c25525a0a8d4ae0bcc1ad80da78c99f
I20260809 22:45:53.044160 28516 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c25525a0a8d4ae0bcc1ad80da78c99f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:53 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:49952
I20260809 22:45:53.044236 29007 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c25525a0a8d4ae0bcc1ad80da78c99f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:53 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:58156
I20260809 22:45:53.045074 28766 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c25525a0a8d4ae0bcc1ad80da78c99f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:53 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:58500
I20260809 22:45:53.053141 28285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:45:53.055833 28285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:53.069830 28514 tablet_service.cc:1467] Processing CreateTablet for tablet 559b97cdb9144a0dbf0f2d87a4ad5881 (DEFAULT_TABLE table=test-table [id=98fa4c2912ce420cb197c6d9f58e43ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:53.070648 28514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 559b97cdb9144a0dbf0f2d87a4ad5881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:53.070587 28764 tablet_service.cc:1467] Processing CreateTablet for tablet 559b97cdb9144a0dbf0f2d87a4ad5881 (DEFAULT_TABLE table=test-table [id=98fa4c2912ce420cb197c6d9f58e43ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:53.071182 28764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 559b97cdb9144a0dbf0f2d87a4ad5881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:53.072029 29006 tablet_service.cc:1467] Processing CreateTablet for tablet 559b97cdb9144a0dbf0f2d87a4ad5881 (DEFAULT_TABLE table=test-table [id=98fa4c2912ce420cb197c6d9f58e43ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:53.072639 29006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 559b97cdb9144a0dbf0f2d87a4ad5881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:53.073266 29258 tablet_replica.cc:330] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: stopping tablet replica
I20260809 22:45:53.073926 29258 raft_consensus.cc:2229] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:53.074695 29253 tablet_replica.cc:330] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: stopping tablet replica
I20260809 22:45:53.075196 29258 raft_consensus.cc:2258] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:53.075775 29253 raft_consensus.cc:2229] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:45:53.080155 29253 raft_consensus.cc:2258] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:53.081262 29258 ts_tablet_manager.cc:1905] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:53.081410 29166 tablet_bootstrap.cc:492] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: Bootstrap starting.
I20260809 22:45:53.083279 29166 tablet_bootstrap.cc:654] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:53.083752 29163 tablet_bootstrap.cc:492] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Bootstrap starting.
I20260809 22:45:53.085424 29163 tablet_bootstrap.cc:654] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:53.085868 29253 ts_tablet_manager.cc:1905] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:53.091427 29256 tablet_replica.cc:330] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: stopping tablet replica
I20260809 22:45:53.094889 29169 tablet_bootstrap.cc:492] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: Bootstrap starting.
I20260809 22:45:53.095661 29256 raft_consensus.cc:2229] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:53.096244 29256 raft_consensus.cc:2258] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:53.096961 29169 tablet_bootstrap.cc:654] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:53.100327 29169 tablet_bootstrap.cc:492] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: No bootstrap required, opened a new log
I20260809 22:45:53.100531 29169 ts_tablet_manager.cc:1397] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 22:45:53.100687 29166 tablet_bootstrap.cc:492] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: No bootstrap required, opened a new log
I20260809 22:45:53.101030 29166 ts_tablet_manager.cc:1397] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.000s
I20260809 22:45:53.101411 29169 raft_consensus.cc:348] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:53.101670 29169 raft_consensus.cc:374] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:53.101758 29169 raft_consensus.cc:729] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291ab60c3be34d47ad0c127139bf1f62, State: Initialized, Role: FOLLOWER
I20260809 22:45:53.102074 29169 consensus_queue.cc:260] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:53.102172 29166 raft_consensus.cc:348] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:53.102571 29166 raft_consensus.cc:374] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:53.102710 29166 raft_consensus.cc:729] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c194e751e34e388c830dc9c2091d47, State: Initialized, Role: FOLLOWER
I20260809 22:45:53.103014 29166 consensus_queue.cc:260] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:53.103225 29258 ts_tablet_manager.cc:1918] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:45:53.103355 29169 ts_tablet_manager.cc:1428] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 22:45:53.103895 29166 ts_tablet_manager.cc:1428] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:45:53.104564 29258 log.cc:1196] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/6c25525a0a8d4ae0bcc1ad80da78c99f
I20260809 22:45:53.104774 29163 tablet_bootstrap.cc:492] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: No bootstrap required, opened a new log
I20260809 22:45:53.104975 29163 ts_tablet_manager.cc:1397] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.006s
I20260809 22:45:53.105155 29258 ts_tablet_manager.cc:1939] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 291ab60c3be34d47ad0c127139bf1f62: Deleting consensus metadata
I20260809 22:45:53.105476 29256 ts_tablet_manager.cc:1905] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:53.105829 29163 raft_consensus.cc:348] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:53.111860 28268 catalog_manager.cc:4841] TS 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377): tablet 6c25525a0a8d4ae0bcc1ad80da78c99f (table test-table [id=c42450533b1443b4b19c3623104479c4]) successfully deleted
I20260809 22:45:53.112026 29163 raft_consensus.cc:374] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:53.112177 29163 raft_consensus.cc:729] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dc84bf63e54c8c991c6c9ea5bcfff0, State: Initialized, Role: FOLLOWER
I20260809 22:45:53.112509 29163 consensus_queue.cc:260] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:53.113293 29163 ts_tablet_manager.cc:1428] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent starting tablet: real 0.008s user 0.002s sys 0.001s
I20260809 22:45:53.125291 29256 ts_tablet_manager.cc:1918] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:45:53.125527 29256 log.cc:1196] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/6c25525a0a8d4ae0bcc1ad80da78c99f
I20260809 22:45:53.126073 29253 ts_tablet_manager.cc:1918] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:45:53.126188 29256 ts_tablet_manager.cc:1939] T 6c25525a0a8d4ae0bcc1ad80da78c99f P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting consensus metadata
I20260809 22:45:53.126304 29253 log.cc:1196] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/6c25525a0a8d4ae0bcc1ad80da78c99f
I20260809 22:45:53.126845 29253 ts_tablet_manager.cc:1939] T 6c25525a0a8d4ae0bcc1ad80da78c99f P f2c194e751e34e388c830dc9c2091d47: Deleting consensus metadata
I20260809 22:45:53.128427 28268 catalog_manager.cc:4841] TS 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855): tablet 6c25525a0a8d4ae0bcc1ad80da78c99f (table test-table [id=c42450533b1443b4b19c3623104479c4]) successfully deleted
I20260809 22:45:53.128698 28271 catalog_manager.cc:4841] TS f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705): tablet 6c25525a0a8d4ae0bcc1ad80da78c99f (table test-table [id=c42450533b1443b4b19c3623104479c4]) successfully deleted
I20260809 22:45:53.232203 29248 raft_consensus.cc:482] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:53.232538 29248 raft_consensus.cc:504] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:53.233497 29248 leader_election.cc:288] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377), 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855)
I20260809 22:45:53.234665 28796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "559b97cdb9144a0dbf0f2d87a4ad5881" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62" is_pre_election: true
I20260809 22:45:53.234992 28796 raft_consensus.cc:2454] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 0.
I20260809 22:45:53.235791 29045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "559b97cdb9144a0dbf0f2d87a4ad5881" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" is_pre_election: true
I20260809 22:45:53.236119 29045 raft_consensus.cc:2454] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 0.
I20260809 22:45:53.236861 28440 leader_election.cc:302] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291ab60c3be34d47ad0c127139bf1f62, f2c194e751e34e388c830dc9c2091d47; no voters:
I20260809 22:45:53.237509 29248 raft_consensus.cc:2781] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:53.237658 29248 raft_consensus.cc:482] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:53.237754 29248 raft_consensus.cc:3037] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:53.240860 29248 raft_consensus.cc:504] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:53.242166 28796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "559b97cdb9144a0dbf0f2d87a4ad5881" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62"
I20260809 22:45:53.242419 28796 raft_consensus.cc:3037] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:53.245721 28796 raft_consensus.cc:2454] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 1.
I20260809 22:45:53.246352 28440 leader_election.cc:302] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291ab60c3be34d47ad0c127139bf1f62, f2c194e751e34e388c830dc9c2091d47; no voters:
I20260809 22:45:53.246769 29177 raft_consensus.cc:2781] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:53.246942 29177 raft_consensus.cc:686] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 1 LEADER]: Becoming Leader. State: Replica: f2c194e751e34e388c830dc9c2091d47, State: Running, Role: LEADER
I20260809 22:45:53.247258 29177 consensus_queue.cc:237] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:53.250725 29248 leader_election.cc:288] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 election: Requested vote from peers 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377), 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855)
I20260809 22:45:53.250824 29045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "559b97cdb9144a0dbf0f2d87a4ad5881" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0"
I20260809 22:45:53.251097 29045 raft_consensus.cc:3037] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:53.254909 29045 raft_consensus.cc:2454] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 1.
I20260809 22:45:53.260499 28284 catalog_manager.cc:5495] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2c194e751e34e388c830dc9c2091d47 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "f2c194e751e34e388c830dc9c2091d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } health_report { overall_health: UNKNOWN } } }
I20260809 22:45:53.294188 29045 raft_consensus.cc:1261] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Refusing update from remote peer f2c194e751e34e388c830dc9c2091d47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:53.295233 29250 consensus_queue.cc:1035] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:53.295989 28796 raft_consensus.cc:1261] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Refusing update from remote peer f2c194e751e34e388c830dc9c2091d47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:53.297245 29250 consensus_queue.cc:1035] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:45:53.331352 29214 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 22:45:54.939790 28284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:54.940060 28284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:54.944820 28284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78: Sending DeleteTablet for 3 replicas of tablet 559b97cdb9144a0dbf0f2d87a4ad5881
I20260809 22:45:54.945858 28764 tablet_service.cc:1514] Processing DeleteTablet for tablet 559b97cdb9144a0dbf0f2d87a4ad5881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:54 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:58500
I20260809 22:45:54.945859 28514 tablet_service.cc:1514] Processing DeleteTablet for tablet 559b97cdb9144a0dbf0f2d87a4ad5881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:54 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:49952
I20260809 22:45:54.945896 29006 tablet_service.cc:1514] Processing DeleteTablet for tablet 559b97cdb9144a0dbf0f2d87a4ad5881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:54 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:58156
I20260809 22:45:54.959110 29324 tablet_replica.cc:330] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: stopping tablet replica
I20260809 22:45:54.959566 29324 raft_consensus.cc:2229] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:54.959990 29324 raft_consensus.cc:2258] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:54.960740 29322 tablet_replica.cc:330] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: stopping tablet replica
I20260809 22:45:54.960944 29322 raft_consensus.cc:2229] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:45:54.961323 29322 raft_consensus.cc:2258] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:54.961380 29324 ts_tablet_manager.cc:1905] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:54.962371 29322 ts_tablet_manager.cc:1905] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:54.962642 29325 tablet_replica.cc:330] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: stopping tablet replica
I20260809 22:45:54.962854 29325 raft_consensus.cc:2229] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:54.963141 29325 raft_consensus.cc:2258] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:54.964160 29325 ts_tablet_manager.cc:1905] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:54.968461 29324 ts_tablet_manager.cc:1918] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:45:54.968631 29324 log.cc:1196] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/559b97cdb9144a0dbf0f2d87a4ad5881
I20260809 22:45:54.969071 29324 ts_tablet_manager.cc:1939] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting consensus metadata
I20260809 22:45:54.969641 29322 ts_tablet_manager.cc:1918] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:45:54.969858 29322 log.cc:1196] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/559b97cdb9144a0dbf0f2d87a4ad5881
I20260809 22:45:54.970372 29322 ts_tablet_manager.cc:1939] T 559b97cdb9144a0dbf0f2d87a4ad5881 P f2c194e751e34e388c830dc9c2091d47: Deleting consensus metadata
I20260809 22:45:54.970427 28268 catalog_manager.cc:4841] TS 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855): tablet 559b97cdb9144a0dbf0f2d87a4ad5881 (table test-table [id=98fa4c2912ce420cb197c6d9f58e43ee]) successfully deleted
I20260809 22:45:54.971081 29325 ts_tablet_manager.cc:1918] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:45:54.971262 29325 log.cc:1196] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/559b97cdb9144a0dbf0f2d87a4ad5881
I20260809 22:45:54.971782 29325 ts_tablet_manager.cc:1939] T 559b97cdb9144a0dbf0f2d87a4ad5881 P 291ab60c3be34d47ad0c127139bf1f62: Deleting consensus metadata
I20260809 22:45:54.971828 28271 catalog_manager.cc:4841] TS f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705): tablet 559b97cdb9144a0dbf0f2d87a4ad5881 (table test-table [id=98fa4c2912ce420cb197c6d9f58e43ee]) successfully deleted
I20260809 22:45:54.972817 28268 catalog_manager.cc:4841] TS 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377): tablet 559b97cdb9144a0dbf0f2d87a4ad5881 (table test-table [id=98fa4c2912ce420cb197c6d9f58e43ee]) successfully deleted
Aug 09 22:45:54 dist-test-slave-pww0 krb5kdc[28215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315554, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:45:55 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315554, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:55.011010 29340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:54.996353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.24.136.62:34923)
0809 22:45:54.996982 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:54.996986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:54.997011 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:45:54.997116 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:54.997120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:54.997188 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:54.997284 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:54.998618 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:54.999433 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.000850 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.001301 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.001414 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:55.005610 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:55.005616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:55.005620 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:55.005668 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:55.008472 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:55.009208 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:55.009212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:55.009214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:55.009372 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:55.009721 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:55.009724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:55.009726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:55.010015 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:55.010194 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:55.010464 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 22:45:55.010655 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":350,"threads_started":1}
I20260809 22:45:55.014803 28284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:45:55.015388 28284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:55.022353 28764 tablet_service.cc:1467] Processing CreateTablet for tablet 0f60f7e29329401e8481a9da8ff62b3e (DEFAULT_TABLE table=test-table [id=d4b1fba4d9d74f20910521f65e569c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:55.022298 28514 tablet_service.cc:1467] Processing CreateTablet for tablet 0f60f7e29329401e8481a9da8ff62b3e (DEFAULT_TABLE table=test-table [id=d4b1fba4d9d74f20910521f65e569c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:55.022814 28764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f60f7e29329401e8481a9da8ff62b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:55.022814 28514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f60f7e29329401e8481a9da8ff62b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:55.022733 29006 tablet_service.cc:1467] Processing CreateTablet for tablet 0f60f7e29329401e8481a9da8ff62b3e (DEFAULT_TABLE table=test-table [id=d4b1fba4d9d74f20910521f65e569c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:55.023211 29006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f60f7e29329401e8481a9da8ff62b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:55.029493 29342 tablet_bootstrap.cc:492] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: Bootstrap starting.
I20260809 22:45:55.030205 29343 tablet_bootstrap.cc:492] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: Bootstrap starting.
I20260809 22:45:55.031518 29342 tablet_bootstrap.cc:654] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:55.031970 29343 tablet_bootstrap.cc:654] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:55.033771 29345 tablet_bootstrap.cc:492] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: Bootstrap starting.
I20260809 22:45:55.034318 29343 tablet_bootstrap.cc:492] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: No bootstrap required, opened a new log
I20260809 22:45:55.034474 29343 ts_tablet_manager.cc:1397] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260809 22:45:55.035126 29342 tablet_bootstrap.cc:492] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: No bootstrap required, opened a new log
I20260809 22:45:55.035274 29342 ts_tablet_manager.cc:1397] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 22:45:55.035307 29343 raft_consensus.cc:348] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.035609 29343 raft_consensus.cc:374] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:55.035692 29345 tablet_bootstrap.cc:654] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:55.035714 29343 raft_consensus.cc:729] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291ab60c3be34d47ad0c127139bf1f62, State: Initialized, Role: FOLLOWER
I20260809 22:45:55.035967 29343 consensus_queue.cc:260] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.036110 29342 raft_consensus.cc:348] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.036381 29342 raft_consensus.cc:374] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:55.036791 29342 raft_consensus.cc:729] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dc84bf63e54c8c991c6c9ea5bcfff0, State: Initialized, Role: FOLLOWER
I20260809 22:45:55.037467 29342 consensus_queue.cc:260] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.038194 29342 ts_tablet_manager.cc:1428] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:45:55.038507 29343 ts_tablet_manager.cc:1428] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 22:45:55.038880 29345 tablet_bootstrap.cc:492] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: No bootstrap required, opened a new log
I20260809 22:45:55.039031 29345 ts_tablet_manager.cc:1397] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 22:45:55.039753 29345 raft_consensus.cc:348] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.039950 29345 raft_consensus.cc:374] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:55.040041 29345 raft_consensus.cc:729] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c194e751e34e388c830dc9c2091d47, State: Initialized, Role: FOLLOWER
I20260809 22:45:55.040287 29345 consensus_queue.cc:260] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.041401 29345 ts_tablet_manager.cc:1428] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:45:55.146140 29314 raft_consensus.cc:482] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:55.146381 29314 raft_consensus.cc:504] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.147125 29314 leader_election.cc:288] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377), 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855)
I20260809 22:45:55.147948 29045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f60f7e29329401e8481a9da8ff62b3e" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" is_pre_election: true
I20260809 22:45:55.147954 28796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f60f7e29329401e8481a9da8ff62b3e" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62" is_pre_election: true
I20260809 22:45:55.148204 29045 raft_consensus.cc:2454] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 0.
I20260809 22:45:55.148211 28796 raft_consensus.cc:2454] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 0.
I20260809 22:45:55.148793 28440 leader_election.cc:302] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52dc84bf63e54c8c991c6c9ea5bcfff0, f2c194e751e34e388c830dc9c2091d47; no voters:
I20260809 22:45:55.149085 29314 raft_consensus.cc:2781] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:55.149235 29314 raft_consensus.cc:482] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:55.149339 29314 raft_consensus.cc:3037] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:55.151966 29314 raft_consensus.cc:504] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.152643 29314 leader_election.cc:288] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 election: Requested vote from peers 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377), 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855)
I20260809 22:45:55.153258 28796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f60f7e29329401e8481a9da8ff62b3e" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62"
I20260809 22:45:55.153354 29045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f60f7e29329401e8481a9da8ff62b3e" candidate_uuid: "f2c194e751e34e388c830dc9c2091d47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0"
I20260809 22:45:55.153484 28796 raft_consensus.cc:3037] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:55.153609 29045 raft_consensus.cc:3037] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:55.155740 28796 raft_consensus.cc:2454] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 1.
I20260809 22:45:55.155802 29045 raft_consensus.cc:2454] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c194e751e34e388c830dc9c2091d47 in term 1.
I20260809 22:45:55.156239 28440 leader_election.cc:302] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291ab60c3be34d47ad0c127139bf1f62, f2c194e751e34e388c830dc9c2091d47; no voters:
I20260809 22:45:55.156548 29314 raft_consensus.cc:2781] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:55.156715 29314 raft_consensus.cc:686] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 1 LEADER]: Becoming Leader. State: Replica: f2c194e751e34e388c830dc9c2091d47, State: Running, Role: LEADER
I20260809 22:45:55.157014 29314 consensus_queue.cc:237] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.160001 28286 catalog_manager.cc:5495] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2c194e751e34e388c830dc9c2091d47 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "f2c194e751e34e388c830dc9c2091d47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } health_report { overall_health: UNKNOWN } } }
I20260809 22:45:55.201606 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:55.195939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49970 (local address 127.24.136.1:39705)
0809 22:45:55.196569 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:55.196573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:55.196594 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:45:55.196655 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:55.196658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:55.196723 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:55.196836 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:55.197294 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.198115 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.199681 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.200128 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.200252 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:55.200762 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:55.200938 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:55.201017 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:55.201195 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 22:45:55.201284 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":317,"threads_started":1}
I20260809 22:45:55.212086 28796 raft_consensus.cc:1261] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Refusing update from remote peer f2c194e751e34e388c830dc9c2091d47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:55.212189 29045 raft_consensus.cc:1261] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Refusing update from remote peer f2c194e751e34e388c830dc9c2091d47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:55.212675 29268 consensus_queue.cc:1035] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:55.213027 29314 consensus_queue.cc:1035] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:55.275069 28286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:55.275342 28286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:55.278491 28286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78: Sending DeleteTablet for 3 replicas of tablet 0f60f7e29329401e8481a9da8ff62b3e
I20260809 22:45:55.279536 28764 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f60f7e29329401e8481a9da8ff62b3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:55 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:58500
I20260809 22:45:55.279516 28514 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f60f7e29329401e8481a9da8ff62b3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:55 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:49952
I20260809 22:45:55.279604 29006 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f60f7e29329401e8481a9da8ff62b3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:55 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:58156
I20260809 22:45:55.279886 29322 tablet_replica.cc:330] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: stopping tablet replica
I20260809 22:45:55.279911 29325 tablet_replica.cc:330] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: stopping tablet replica
I20260809 22:45:55.280109 29322 raft_consensus.cc:2229] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:45:55.280103 29325 raft_consensus.cc:2229] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:55.280274 29324 tablet_replica.cc:330] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: stopping tablet replica
I20260809 22:45:55.280341 29325 raft_consensus.cc:2258] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:55.280452 29322 raft_consensus.cc:2258] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:55.280490 29324 raft_consensus.cc:2229] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:55.280712 29324 raft_consensus.cc:2258] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:55.281574 29325 ts_tablet_manager.cc:1905] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:55.281572 29322 ts_tablet_manager.cc:1905] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:55.281802 29324 ts_tablet_manager.cc:1905] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:55.281728 28286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:45:55.282648 28286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:55.288745 29324 ts_tablet_manager.cc:1918] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:45:55.289016 29324 log.cc:1196] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/0f60f7e29329401e8481a9da8ff62b3e
I20260809 22:45:55.289530 29324 ts_tablet_manager.cc:1939] T 0f60f7e29329401e8481a9da8ff62b3e P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting consensus metadata
I20260809 22:45:55.290143 29325 ts_tablet_manager.cc:1918] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:45:55.290375 29325 log.cc:1196] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/0f60f7e29329401e8481a9da8ff62b3e
I20260809 22:45:55.290653 29322 ts_tablet_manager.cc:1918] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:45:55.290831 29322 log.cc:1196] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/0f60f7e29329401e8481a9da8ff62b3e
I20260809 22:45:55.290807 28268 catalog_manager.cc:4841] TS 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855): tablet 0f60f7e29329401e8481a9da8ff62b3e (table test-table [id=d4b1fba4d9d74f20910521f65e569c55]) successfully deleted
I20260809 22:45:55.291617 29325 ts_tablet_manager.cc:1939] T 0f60f7e29329401e8481a9da8ff62b3e P 291ab60c3be34d47ad0c127139bf1f62: Deleting consensus metadata
I20260809 22:45:55.292615 29322 ts_tablet_manager.cc:1939] T 0f60f7e29329401e8481a9da8ff62b3e P f2c194e751e34e388c830dc9c2091d47: Deleting consensus metadata
I20260809 22:45:55.292615 28514 tablet_service.cc:1467] Processing CreateTablet for tablet 4594f78723a741328c681075364ee57c (DEFAULT_TABLE table=test-table [id=f33914254801457cb6466512f45b8c3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:55.292968 29006 tablet_service.cc:1467] Processing CreateTablet for tablet 4594f78723a741328c681075364ee57c (DEFAULT_TABLE table=test-table [id=f33914254801457cb6466512f45b8c3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:55.293125 28514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4594f78723a741328c681075364ee57c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:55.293200 28764 tablet_service.cc:1467] Processing CreateTablet for tablet 4594f78723a741328c681075364ee57c (DEFAULT_TABLE table=test-table [id=f33914254801457cb6466512f45b8c3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:55.293431 29006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4594f78723a741328c681075364ee57c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:55.293605 28764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4594f78723a741328c681075364ee57c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:55.293947 28271 catalog_manager.cc:4841] TS f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705): tablet 0f60f7e29329401e8481a9da8ff62b3e (table test-table [id=d4b1fba4d9d74f20910521f65e569c55]) successfully deleted
I20260809 22:45:55.294219 28268 catalog_manager.cc:4841] TS 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377): tablet 0f60f7e29329401e8481a9da8ff62b3e (table test-table [id=d4b1fba4d9d74f20910521f65e569c55]) successfully deleted
I20260809 22:45:55.298653 29345 tablet_bootstrap.cc:492] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: Bootstrap starting.
I20260809 22:45:55.300139 29343 tablet_bootstrap.cc:492] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: Bootstrap starting.
I20260809 22:45:55.300565 29345 tablet_bootstrap.cc:654] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:55.300825 29342 tablet_bootstrap.cc:492] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: Bootstrap starting.
I20260809 22:45:55.301872 29343 tablet_bootstrap.cc:654] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:55.302346 29342 tablet_bootstrap.cc:654] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:55.303490 29345 tablet_bootstrap.cc:492] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: No bootstrap required, opened a new log
I20260809 22:45:55.303678 29345 ts_tablet_manager.cc:1397] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 22:45:55.304502 29342 tablet_bootstrap.cc:492] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: No bootstrap required, opened a new log
I20260809 22:45:55.304433 29345 raft_consensus.cc:348] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.304625 29345 raft_consensus.cc:374] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:55.304663 29342 ts_tablet_manager.cc:1397] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260809 22:45:55.304692 29345 raft_consensus.cc:729] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c194e751e34e388c830dc9c2091d47, State: Initialized, Role: FOLLOWER
I20260809 22:45:55.304973 29345 consensus_queue.cc:260] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.305282 29342 raft_consensus.cc:348] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.305469 29343 tablet_bootstrap.cc:492] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: No bootstrap required, opened a new log
I20260809 22:45:55.305546 29342 raft_consensus.cc:374] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:55.305621 29342 raft_consensus.cc:729] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dc84bf63e54c8c991c6c9ea5bcfff0, State: Initialized, Role: FOLLOWER
I20260809 22:45:55.305634 29343 ts_tablet_manager.cc:1397] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 22:45:55.305672 29345 ts_tablet_manager.cc:1428] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:45:55.305847 29342 consensus_queue.cc:260] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.306368 29342 ts_tablet_manager.cc:1428] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 22:45:55.306347 29343 raft_consensus.cc:348] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.306584 29343 raft_consensus.cc:374] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:55.306658 29343 raft_consensus.cc:729] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291ab60c3be34d47ad0c127139bf1f62, State: Initialized, Role: FOLLOWER
I20260809 22:45:55.306900 29343 consensus_queue.cc:260] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.307533 29343 ts_tablet_manager.cc:1428] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 22:45:55.573195 29350 raft_consensus.cc:482] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:55.573586 29282 raft_consensus.cc:482] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:55.573511 29350 raft_consensus.cc:504] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.573803 29282 raft_consensus.cc:504] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.574546 29282 leader_election.cc:288] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705), 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377)
I20260809 22:45:55.575261 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4594f78723a741328c681075364ee57c" candidate_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62" is_pre_election: true
I20260809 22:45:55.575393 29350 leader_election.cc:288] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705), 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855)
I20260809 22:45:55.575260 28555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4594f78723a741328c681075364ee57c" candidate_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c194e751e34e388c830dc9c2091d47" is_pre_election: true
I20260809 22:45:55.575549 28798 raft_consensus.cc:2454] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52dc84bf63e54c8c991c6c9ea5bcfff0 in term 0.
I20260809 22:45:55.575877 28555 raft_consensus.cc:2454] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52dc84bf63e54c8c991c6c9ea5bcfff0 in term 0.
I20260809 22:45:55.576157 28919 leader_election.cc:302] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291ab60c3be34d47ad0c127139bf1f62, 52dc84bf63e54c8c991c6c9ea5bcfff0; no voters:
I20260809 22:45:55.576570 29282 raft_consensus.cc:2781] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:55.576722 29282 raft_consensus.cc:482] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:55.576829 29282 raft_consensus.cc:3037] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:55.580065 29282 raft_consensus.cc:504] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.581344 28555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4594f78723a741328c681075364ee57c" candidate_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c194e751e34e388c830dc9c2091d47"
I20260809 22:45:55.581367 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4594f78723a741328c681075364ee57c" candidate_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "291ab60c3be34d47ad0c127139bf1f62"
I20260809 22:45:55.581614 28798 raft_consensus.cc:3037] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:55.581676 28555 raft_consensus.cc:3037] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:55.582466 29282 leader_election.cc:288] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [CANDIDATE]: Term 1 election: Requested vote from peers f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705), 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377)
I20260809 22:45:55.584947 28555 raft_consensus.cc:2454] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52dc84bf63e54c8c991c6c9ea5bcfff0 in term 1.
I20260809 22:45:55.584944 28798 raft_consensus.cc:2454] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52dc84bf63e54c8c991c6c9ea5bcfff0 in term 1.
I20260809 22:45:55.585525 28925 leader_election.cc:302] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 291ab60c3be34d47ad0c127139bf1f62, 52dc84bf63e54c8c991c6c9ea5bcfff0, f2c194e751e34e388c830dc9c2091d47; no voters:
I20260809 22:45:55.585973 29282 raft_consensus.cc:2781] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:55.586252 29282 raft_consensus.cc:686] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 LEADER]: Becoming Leader. State: Replica: 52dc84bf63e54c8c991c6c9ea5bcfff0, State: Running, Role: LEADER
I20260809 22:45:55.586400 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:55.576970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:32779 (local address 127.24.136.1:39705)
0809 22:45:55.577182 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:55.577189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:55.577782 (+ 593us) server_negotiation.cc:407] Connection header received
0809 22:45:55.578116 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:55.578122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:55.578204 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:55.578267 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:55.579052 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.580210 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.583263 (+ 3053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.584746 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.584938 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:55.585016 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:55.585810 (+ 794us) server_negotiation.cc:299] Negotiation successful
0809 22:45:55.585964 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:45:55.586637 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:55.576144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:57041 (local address 127.24.136.3:44855)
0809 22:45:55.576793 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:55.576799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:55.578089 (+ 1290us) server_negotiation.cc:407] Connection header received
0809 22:45:55.578347 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:55.578353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:55.578437 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:55.578549 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:55.579312 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.580363 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.583397 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.584798 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.584936 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:55.585016 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:55.585818 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 22:45:55.585977 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":593,"threads_started":1}
I20260809 22:45:55.586927 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:55.576495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44855 (local address 127.24.136.2:57041)
0809 22:45:55.577999 (+ 1504us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:55.578015 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:55.578188 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:55.578597 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:55.578601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:55.578626 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:55.579095 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:55.579105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.580519 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.580524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:55.583141 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:55.583178 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.585286 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.585294 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:55.585523 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:55.585640 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 22:45:55.586314 (+ 674us) client_negotiation.cc:240] Negotiation successful
0809 22:45:55.586485 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1391,"mutex_wait_us":352,"thread_start_us":879,"threads_started":1}
I20260809 22:45:55.587175 29282 consensus_queue.cc:237] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } }
I20260809 22:45:55.587494 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:55.576020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.24.136.2:32779)
0809 22:45:55.577665 (+ 1645us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:55.577686 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:55.577920 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:55.578457 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:55.578460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:55.578482 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:55.578845 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:55.578857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.580461 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.580467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:55.582899 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:55.582918 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.585303 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.585309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:55.585520 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:55.585638 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 22:45:55.586689 (+ 1051us) client_negotiation.cc:240] Negotiation successful
0809 22:45:55.586847 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1508,"mutex_wait_us":75,"thread_start_us":1713,"threads_started":1}
I20260809 22:45:55.587783 29045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4594f78723a741328c681075364ee57c" candidate_uuid: "291ab60c3be34d47ad0c127139bf1f62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" is_pre_election: true
I20260809 22:45:55.588289 28555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4594f78723a741328c681075364ee57c" candidate_uuid: "291ab60c3be34d47ad0c127139bf1f62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c194e751e34e388c830dc9c2091d47" is_pre_election: true
I20260809 22:45:55.589272 28555 raft_consensus.cc:2379] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 291ab60c3be34d47ad0c127139bf1f62 in current term 1: Already voted for candidate 52dc84bf63e54c8c991c6c9ea5bcfff0 in this term.
I20260809 22:45:55.589869 28682 leader_election.cc:302] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 291ab60c3be34d47ad0c127139bf1f62; no voters: 52dc84bf63e54c8c991c6c9ea5bcfff0, f2c194e751e34e388c830dc9c2091d47
I20260809 22:45:55.590260 29350 raft_consensus.cc:2726] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 22:45:55.590319 28284 catalog_manager.cc:5495] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52dc84bf63e54c8c991c6c9ea5bcfff0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44855 } health_report { overall_health: HEALTHY } } }
I20260809 22:45:55.614099 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:55.608911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.24.136.3:44855)
0809 22:45:55.609192 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:55.609199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:55.609299 (+ 100us) server_negotiation.cc:407] Connection header received
0809 22:45:55.609532 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:55.609536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:55.609608 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:55.609695 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:55.610100 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.610879 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.612435 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.612863 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.612976 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:55.613420 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:55.613531 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:55.613605 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:55.613765 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 22:45:55.613851 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 22:45:55.630558 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:55.624044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:41483 (local address 127.24.136.3:44855)
0809 22:45:55.624242 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:55.624249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:55.624829 (+ 580us) server_negotiation.cc:407] Connection header received
0809 22:45:55.625062 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:55.625067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:55.625171 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:55.625260 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:55.625776 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.626546 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.628575 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.629017 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.629137 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:55.629741 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:55.629854 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:55.629929 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:55.630176 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 22:45:55.630272 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":57}
I20260809 22:45:55.630750 29381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:55.624032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44855 (local address 127.24.136.3:41483)
0809 22:45:55.624716 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:55.624736 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:55.624893 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:55.625286 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:55.625291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:55.625304 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:45:55.625625 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:55.625637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.626689 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.626696 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:55.628382 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:55.628398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:55.629206 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:55.629213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:55.629462 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:55.629515 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:45:55.629994 (+ 479us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:45:55.630009 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 22:45:55.630109 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 22:45:55.630295 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":33,"thread_start_us":361,"threads_started":1}
I20260809 22:45:55.639577 28798 raft_consensus.cc:1261] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Refusing update from remote peer 52dc84bf63e54c8c991c6c9ea5bcfff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:55.640188 29282 consensus_queue.cc:1035] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "291ab60c3be34d47ad0c127139bf1f62" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:55.640844 28555 raft_consensus.cc:1261] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Refusing update from remote peer 52dc84bf63e54c8c991c6c9ea5bcfff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:55.641737 29275 consensus_queue.cc:1035] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2c194e751e34e388c830dc9c2091d47" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:45:55.664067 29364 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 22:45:57.274645 28284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:57.274921 28284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:57.279587 28284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8364e77f644a40ec833c0804b7815b78: Sending DeleteTablet for 3 replicas of tablet 4594f78723a741328c681075364ee57c
I20260809 22:45:57.280625 28514 tablet_service.cc:1514] Processing DeleteTablet for tablet 4594f78723a741328c681075364ee57c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:57 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:49952
I20260809 22:45:57.280619 28764 tablet_service.cc:1514] Processing DeleteTablet for tablet 4594f78723a741328c681075364ee57c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:57 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:58500
I20260809 22:45:57.280647 29007 tablet_service.cc:1514] Processing DeleteTablet for tablet 4594f78723a741328c681075364ee57c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:57 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:58156
I20260809 22:45:57.281601 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28418
I20260809 22:45:57.296303 29432 tablet_replica.cc:330] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: stopping tablet replica
I20260809 22:45:57.296568 29432 raft_consensus.cc:2229] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:45:57.297335 29432 raft_consensus.cc:2258] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:57.298704 29432 ts_tablet_manager.cc:1905] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:57.307132 29432 ts_tablet_manager.cc:1918] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:45:57.307375 29432 log.cc:1196] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/4594f78723a741328c681075364ee57c
I20260809 22:45:57.307909 29436 tablet_replica.cc:330] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: stopping tablet replica
I20260809 22:45:57.308151 29431 tablet_replica.cc:330] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: stopping tablet replica
I20260809 22:45:57.308126 29436 raft_consensus.cc:2229] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 09 22:45:57 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315554, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:45:57.308352 29431 raft_consensus.cc:2229] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:57.308435 29436 raft_consensus.cc:2258] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:57.308591 29431 raft_consensus.cc:2258] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:57.308894 29432 ts_tablet_manager.cc:1939] T 4594f78723a741328c681075364ee57c P 52dc84bf63e54c8c991c6c9ea5bcfff0: Deleting consensus metadata
I20260809 22:45:57.309620 29431 ts_tablet_manager.cc:1905] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:57.309610 29436 ts_tablet_manager.cc:1905] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:57.310457 28268 catalog_manager.cc:4841] TS 52dc84bf63e54c8c991c6c9ea5bcfff0 (127.24.136.3:44855): tablet 4594f78723a741328c681075364ee57c (table test-table [id=f33914254801457cb6466512f45b8c3b]) successfully deleted
I20260809 22:45:57.316344 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:57.284385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.24.136.1:39705)
0809 22:45:57.290040 (+ 5655us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:57.290048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:57.294203 (+ 4155us) server_negotiation.cc:407] Connection header received
0809 22:45:57.294798 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:57.294805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:57.294905 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:57.295010 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:57.297114 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:57.299634 (+ 2520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:57.302032 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:57.303015 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:57.303196 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:57.309688 (+ 6492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:57.309711 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:57.309735 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:57.309783 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:57.313285 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:57.314073 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:57.314082 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:57.314084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:57.314267 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:57.314676 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:57.314679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:57.314681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:57.315062 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:57.315382 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:57.315745 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 22:45:57.315975 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5473,"thread_start_us":5281,"threads_started":1}
I20260809 22:45:57.316649 29436 ts_tablet_manager.cc:1918] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:45:57.316811 29436 log.cc:1196] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/4594f78723a741328c681075364ee57c
I20260809 22:45:57.316903 29431 ts_tablet_manager.cc:1918] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:45:57.317102 29431 log.cc:1196] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/4594f78723a741328c681075364ee57c
W20260809 22:45:57.317217 28641 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49980
I20260809 22:45:57.317324 29436 ts_tablet_manager.cc:1939] T 4594f78723a741328c681075364ee57c P 291ab60c3be34d47ad0c127139bf1f62: Deleting consensus metadata
I20260809 22:45:57.317595 29431 ts_tablet_manager.cc:1939] T 4594f78723a741328c681075364ee57c P f2c194e751e34e388c830dc9c2091d47: Deleting consensus metadata
W20260809 22:45:57.317939 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28418: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:45:57.318179 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 28418
I20260809 22:45:57.327526 28268 catalog_manager.cc:4841] TS 291ab60c3be34d47ad0c127139bf1f62 (127.24.136.2:42377): tablet 4594f78723a741328c681075364ee57c (table test-table [id=f33914254801457cb6466512f45b8c3b]) successfully deleted
W20260809 22:45:57.352280 28271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.136.1:39705 (error 108)
I20260809 22:45:57.352458 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28652
I20260809 22:45:57.354802 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:57.353837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.0.0.1:49994)
0809 22:45:57.354409 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:57.354564 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":294,"threads_started":1}
W20260809 22:45:57.355022 28271 catalog_manager.cc:4584] TS f2c194e751e34e388c830dc9c2091d47 (127.24.136.1:39705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4594f78723a741328c681075364ee57c: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39705: connect: Connection refused (error 111)
Aug 09 22:45:57 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315554, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:45:57.368381 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:57.353312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58508 (local address 127.24.136.2:42377)
0809 22:45:57.353782 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:57.353786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:57.353806 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:45:57.353864 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:57.353868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:57.353929 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:57.354052 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:57.355650 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:57.356446 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:57.357727 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:57.358145 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:57.358248 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:57.362358 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:57.362376 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:57.362393 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:57.362433 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:57.365442 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:57.366208 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:57.366219 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:57.366222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:57.366410 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:57.366860 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:57.366864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:57.366866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:57.367274 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:57.367581 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:57.367850 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 22:45:57.368070 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":233,"threads_started":1}
W20260809 22:45:57.369177 28871 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58508
W20260809 22:45:57.369856 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28652: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:45:57.370009 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 28652
I20260809 22:45:57.402199 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28886
I20260809 22:45:57.408492 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:57.408015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.0.0.1:49996)
0809 22:45:57.408243 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:57.408339 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 22:45:57.409657 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:57.409221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.0.0.1:50008)
0809 22:45:57.409406 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:57.409504 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
Aug 09 22:45:57 dist-test-slave-pww0 krb5kdc[28215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315554, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:45:57.418660 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:57.403164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.24.136.3:44855)
0809 22:45:57.403745 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:57.403750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:57.403768 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:45:57.403822 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:57.403825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:57.403894 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:57.403989 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:57.405404 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:57.406140 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:57.407433 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:57.407978 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:57.408081 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:57.412574 (+ 4493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:57.412594 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:57.412611 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:57.412653 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:57.415625 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:57.416430 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:57.416438 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:57.416440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:57.416615 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:57.417036 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:57.417040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:57.417042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:57.417437 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:57.417740 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:57.418073 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 22:45:57.418323 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":289,"threads_started":1}
W20260809 22:45:57.419450 29116 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58170
W20260809 22:45:57.420277 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28886: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:45:57.420492 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 28886
I20260809 22:45:57.456199 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28244
I20260809 22:45:57.457459 28364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:57.577809 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:57.577342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.0.0.1:50020)
0809 22:45:57.577528 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:57.577638 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 22:45:57.577870 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 28244
2026-08-09T22:45:57Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9285 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:45:57 dist-test-slave-pww0 krb5kdc[29451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:45:57 dist-test-slave-pww0 krb5kdc[29451](info): set up 2 sockets
Aug 09 22:45:57 dist-test-slave-pww0 krb5kdc[29451](info): commencing operation
krb5kdc: starting...
W20260809 22:45:58.828199 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:45:58 dist-test-slave-pww0 krb5kdc[29451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:45:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:45:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:45:59.055792 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:46055
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:46055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:59.410558 29471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:59.410905 29471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:59.411043 29471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:59.418526 29471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:45:59.418675 29471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:59.418747 29471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:59.418840 29471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:45:59.418902 29471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:45:59.429183 29471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:46055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:46055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:45:59.430644 29471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:59.432044 29471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:59.444803 29480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:59.444936 29481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:59.444919 29483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:59.445272 29471 server_base.cc:1034] running on GCE node
I20260809 22:45:59.446244 29471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:59.447942 29471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:59.449208 29471 hybrid_clock.cc:648] HybridClock initialized: now 1786315559449140 us; error 43 us; skew 500 ppm
I20260809 22:45:59.449667 29471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:59 dist-test-slave-pww0 krb5kdc[29451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315559, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:59.453115 29471 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:45:59.454816 29471 webserver.cc:460] Webserver started at http://127.24.136.62:35087/ using document root <none> and password file <none>
I20260809 22:45:59.455626 29471 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:59.455746 29471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:59.456069 29471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:45:59.459304 29471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "5b6bc5b96e984e1488d8eca9ef320cb4"
format_stamp: "Formatted at 2026-08-09 22:45:59 on dist-test-slave-pww0"
server_key: "af3ceb33195d70824345380421816e40"
server_key_iv: "f2568495e8f11e755872257c2efb4cc1"
server_key_version: "encryptionkey@0"
I20260809 22:45:59.460224 29471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "5b6bc5b96e984e1488d8eca9ef320cb4"
format_stamp: "Formatted at 2026-08-09 22:45:59 on dist-test-slave-pww0"
server_key: "af3ceb33195d70824345380421816e40"
server_key_iv: "f2568495e8f11e755872257c2efb4cc1"
server_key_version: "encryptionkey@0"
I20260809 22:45:59.465950 29471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:45:59.469874 29497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:59.471412 29471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:45:59.471691 29471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "5b6bc5b96e984e1488d8eca9ef320cb4"
format_stamp: "Formatted at 2026-08-09 22:45:59 on dist-test-slave-pww0"
server_key: "af3ceb33195d70824345380421816e40"
server_key_iv: "f2568495e8f11e755872257c2efb4cc1"
server_key_version: "encryptionkey@0"
I20260809 22:45:59.471900 29471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:59.499250 29471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:59.503942 29471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:59.504227 29471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:59.522401 29471 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:46055
I20260809 22:45:59.522413 29583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:46055 every 8 connection(s)
I20260809 22:45:59.523849 29471 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:45:59.527837 29584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:59.528930 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 29471
I20260809 22:45:59.529179 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:45:59.529881 25120 external_mini_cluster.cc:1428] Setting key 8516c11933775aa8696f122e0bab446a
I20260809 22:45:59.540644 29584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4: Bootstrap starting.
I20260809 22:45:59.544770 29584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4: Neither blocks nor log segments found. Creating new log.
Aug 09 22:45:59 dist-test-slave-pww0 krb5kdc[29451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:45:59.546005 29584 log.cc:824] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:59.549423 29584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4: No bootstrap required, opened a new log
I20260809 22:45:59.555049 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:59.532144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53244 (local address 127.24.136.62:46055)
0809 22:45:59.533097 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:59.533110 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:59.533167 (+ 57us) server_negotiation.cc:407] Connection header received
0809 22:45:59.535149 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:59.535182 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:59.535696 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:59.536063 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:59.538042 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:59.539508 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:59.541297 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:59.542032 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:59.542196 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:59.547143 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:59.547155 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:59.547178 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:59.547238 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:59.550485 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:59.551492 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:59.551502 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:59.551513 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:59.551734 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:59.552154 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:59.552158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:59.552163 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:59.552773 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:59.553005 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:59.553489 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 22:45:59.553912 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":425,"threads_started":1}
I20260809 22:45:59.558152 29584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6bc5b96e984e1488d8eca9ef320cb4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46055 } }
I20260809 22:45:59.558521 29584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:59.558622 29584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b6bc5b96e984e1488d8eca9ef320cb4, State: Initialized, Role: FOLLOWER
I20260809 22:45:59.559216 29584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6bc5b96e984e1488d8eca9ef320cb4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46055 } }
I20260809 22:45:59.559437 29584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:45:59.559589 29584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:45:59.559748 29584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:59.561967 29584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6bc5b96e984e1488d8eca9ef320cb4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46055 } }
I20260809 22:45:59.562463 29584 leader_election.cc:302] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b6bc5b96e984e1488d8eca9ef320cb4; no voters:
I20260809 22:45:59.563028 29584 leader_election.cc:288] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:45:59.563175 29592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:59.563836 29592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [term 1 LEADER]: Becoming Leader. State: Replica: 5b6bc5b96e984e1488d8eca9ef320cb4, State: Running, Role: LEADER
I20260809 22:45:59.564436 29592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6bc5b96e984e1488d8eca9ef320cb4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46055 } }
I20260809 22:45:59.565073 29584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:45:59.566879 29593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b6bc5b96e984e1488d8eca9ef320cb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6bc5b96e984e1488d8eca9ef320cb4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46055 } } }
I20260809 22:45:59.567209 29593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:59.569271 29594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b6bc5b96e984e1488d8eca9ef320cb4. Latest consensus state: current_term: 1 leader_uuid: "5b6bc5b96e984e1488d8eca9ef320cb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6bc5b96e984e1488d8eca9ef320cb4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46055 } } }
I20260809 22:45:59.569531 29594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:59.571102 29606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:45:59.576583 29606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:45:59.587116 29606 catalog_manager.cc:1347] Generated new cluster ID: d77372c19d0c43b3a6093f7c70c6c23b
I20260809 22:45:59.587249 29606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:45:59.603214 29606 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:45:59.605334 29606 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:45:59.624259 29606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b6bc5b96e984e1488d8eca9ef320cb4: Generated new TSK 0
I20260809 22:45:59.625479 29606 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:45:59.689615 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:46055
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:00.038189 29631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:00.038512 29631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:00.038657 29631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:00.046401 29631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:00.046563 29631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:00.046723 29631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:00.057554 29631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:00.058966 29631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:00.060433 29631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:00.074585 29645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:00.074690 29642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:00.074630 29648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:00.075249 29631 server_base.cc:1034] running on GCE node
I20260809 22:46:00.076097 29631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:00.077170 29631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:00.078383 29631 hybrid_clock.cc:648] HybridClock initialized: now 1786315560078327 us; error 52 us; skew 500 ppm
I20260809 22:46:00.078724 29631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:00 dist-test-slave-pww0 krb5kdc[29451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315560, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:00.082003 29631 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:46:00.083683 29631 webserver.cc:460] Webserver started at http://127.24.136.1:40183/ using document root <none> and password file <none>
I20260809 22:46:00.084422 29631 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:00.084550 29631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:00.084877 29631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:00.088008 29631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "abffd7882f294e2f84288f89c97b153e"
format_stamp: "Formatted at 2026-08-09 22:46:00 on dist-test-slave-pww0"
server_key: "17267b4ee4b9e0f374a26161a8663834"
server_key_iv: "3aef4e24ca7b881cdf2ac3e13d663414"
server_key_version: "encryptionkey@0"
I20260809 22:46:00.088835 29631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "abffd7882f294e2f84288f89c97b153e"
format_stamp: "Formatted at 2026-08-09 22:46:00 on dist-test-slave-pww0"
server_key: "17267b4ee4b9e0f374a26161a8663834"
server_key_iv: "3aef4e24ca7b881cdf2ac3e13d663414"
server_key_version: "encryptionkey@0"
I20260809 22:46:00.094304 29631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:46:00.098071 29661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:00.099714 29631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:46:00.099900 29631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "abffd7882f294e2f84288f89c97b153e"
format_stamp: "Formatted at 2026-08-09 22:46:00 on dist-test-slave-pww0"
server_key: "17267b4ee4b9e0f374a26161a8663834"
server_key_iv: "3aef4e24ca7b881cdf2ac3e13d663414"
server_key_version: "encryptionkey@0"
I20260809 22:46:00.100118 29631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:00.124378 29631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:00.128916 29631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:00.129223 29631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:00.130307 29631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:00.131803 29631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:00.131883 29631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:00.131968 29631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:00.132011 29631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:00.164366 29631 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:39205
I20260809 22:46:00.164446 29873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:39205 every 8 connection(s)
I20260809 22:46:00.165863 29631 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:00.174160 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 29631
I20260809 22:46:00.174407 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:00.175247 25120 external_mini_cluster.cc:1428] Setting key 3d0c5164ce93cad95e884b4b824c121e
Aug 09 22:46:00 dist-test-slave-pww0 krb5kdc[29451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315560, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:00.188522 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:00.168645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:53433 (local address 127.24.136.62:46055)
0809 22:46:00.169108 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:00.169115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:00.170253 (+ 1138us) server_negotiation.cc:407] Connection header received
0809 22:46:00.172274 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:00.172279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:00.172352 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:00.172439 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:00.174136 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:00.175317 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:00.177238 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:00.177808 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:00.177945 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:00.182070 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:00.182076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:00.182079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:00.182117 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:00.184846 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:00.185873 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:00.185879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:00.185881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:00.186020 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:00.186547 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:00.186552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:00.186555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:00.186926 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:00.187083 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:00.187893 (+ 810us) server_negotiation.cc:299] Negotiation successful
0809 22:46:00.188133 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":204,"threads_started":1}
I20260809 22:46:00.189334 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:00.169238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:46055 (local address 127.24.136.1:53433)
0809 22:46:00.170059 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:00.170107 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:00.171970 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:00.172756 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:00.172765 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:00.173329 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:00.173955 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:00.173973 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:00.175526 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:00.175532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:00.177077 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:00.177094 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:00.178053 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:00.178058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:00.178278 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:00.179167 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:00.179202 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:00.181774 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:00.185019 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:00.185030 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:00.185052 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:00.185709 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:00.186137 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:00.186142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:00.186145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:00.186381 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:00.187043 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:00.187050 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:00.187411 (+ 361us) client_negotiation.cc:769] Sending connection context
0809 22:46:00.187722 (+ 311us) client_negotiation.cc:240] Negotiation successful
0809 22:46:00.188154 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":303,"threads_started":1}
I20260809 22:46:00.190969 29875 heartbeater.cc:347] Connected to a master server at 127.24.136.62:46055
I20260809 22:46:00.191286 29875 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:00.192054 29875 heartbeater.cc:511] Master 127.24.136.62:46055 requested a full tablet report, sending...
I20260809 22:46:00.194674 29527 ts_manager.cc:194] Registered new tserver with Master: abffd7882f294e2f84288f89c97b153e (127.24.136.1:39205)
I20260809 22:46:00.196931 29527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:53433
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:46:00.227401 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:46055
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:00.577304 29882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:00.577621 29882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:00.577746 29882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:00.585259 29882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:00.585397 29882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:00.585551 29882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:00.595655 29882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:00.597033 29882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:00.598471 29882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:00.612438 29888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:00.612417 29896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:00.612375 29889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:00.613269 29882 server_base.cc:1034] running on GCE node
I20260809 22:46:00.613953 29882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:00.615043 29882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:00.616288 29882 hybrid_clock.cc:648] HybridClock initialized: now 1786315560616234 us; error 48 us; skew 500 ppm
I20260809 22:46:00.616650 29882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:00 dist-test-slave-pww0 krb5kdc[29451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315560, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:00.620296 29882 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:46:00.621930 29882 webserver.cc:460] Webserver started at http://127.24.136.2:36299/ using document root <none> and password file <none>
I20260809 22:46:00.622656 29882 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:00.622798 29882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:00.623111 29882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:00.626169 29882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "d1a04dc5147d4759839dfd035fe2fc5d"
format_stamp: "Formatted at 2026-08-09 22:46:00 on dist-test-slave-pww0"
server_key: "93716033ea58d713fff8de0eb5506cb6"
server_key_iv: "1f7a0c141f3fd3ae6539fb9dd1cbda33"
server_key_version: "encryptionkey@0"
I20260809 22:46:00.626979 29882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "d1a04dc5147d4759839dfd035fe2fc5d"
format_stamp: "Formatted at 2026-08-09 22:46:00 on dist-test-slave-pww0"
server_key: "93716033ea58d713fff8de0eb5506cb6"
server_key_iv: "1f7a0c141f3fd3ae6539fb9dd1cbda33"
server_key_version: "encryptionkey@0"
I20260809 22:46:00.632810 29882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 22:46:00.636591 29911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:00.638026 29882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:00.638199 29882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "d1a04dc5147d4759839dfd035fe2fc5d"
format_stamp: "Formatted at 2026-08-09 22:46:00 on dist-test-slave-pww0"
server_key: "93716033ea58d713fff8de0eb5506cb6"
server_key_iv: "1f7a0c141f3fd3ae6539fb9dd1cbda33"
server_key_version: "encryptionkey@0"
I20260809 22:46:00.638403 29882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:00.679245 29882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:00.684412 29882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:00.684700 29882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:00.685706 29882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:00.687214 29882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:00.687294 29882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:00.687384 29882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:00.687438 29882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:00.718511 29882 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:45137
I20260809 22:46:00.718561 30111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:45137 every 8 connection(s)
I20260809 22:46:00.719918 29882 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:00.721182 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 29882
I20260809 22:46:00.721369 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:00.721773 25120 external_mini_cluster.cc:1428] Setting key b95b4a19c072fd39d5d2f4249f7a469c
Aug 09 22:46:00 dist-test-slave-pww0 krb5kdc[29451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315560, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:00.743031 30117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:00.722760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:43113 (local address 127.24.136.62:46055)
0809 22:46:00.723231 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:00.723237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:00.724451 (+ 1214us) server_negotiation.cc:407] Connection header received
0809 22:46:00.726321 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:00.726326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:00.726414 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:00.726514 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:00.728285 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:00.729527 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:00.731934 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:00.732663 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:00.732798 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:00.736593 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:00.736599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:00.736603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:00.736643 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:00.739356 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:00.740276 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:00.740282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:00.740284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:00.740438 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:00.740924 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:00.740928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:00.740930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:00.741236 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:00.741383 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:00.742266 (+ 883us) server_negotiation.cc:299] Negotiation successful
0809 22:46:00.742525 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":227,"threads_started":1}
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
I20260809 22:46:00.744120 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:00.723446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:46055 (local address 127.24.136.2:43113)
0809 22:46:00.724296 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:00.724343 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:00.726037 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:00.726789 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:00.726797 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:00.727360 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:00.728068 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:00.728093 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:00.729726 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:00.729731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:00.731722 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:00.731741 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:00.732932 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:00.732941 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:00.733196 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:00.733891 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:00.733919 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:00.736288 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:00.739577 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:00.739588 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:00.739608 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:00.740119 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:00.740560 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:00.740566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:00.740568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:00.740795 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:00.741386 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:00.741396 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:00.741813 (+ 417us) client_negotiation.cc:769] Sending connection context
0809 22:46:00.742106 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 22:46:00.742727 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":361,"threads_started":1}
I20260809 22:46:00.745821 30113 heartbeater.cc:347] Connected to a master server at 127.24.136.62:46055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
I20260809 22:46:00.746232 30113 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:00.747113 30113 heartbeater.cc:511] Master 127.24.136.62:46055 requested a full tablet report, sending...
I20260809 22:46:00.748963 29527 ts_manager.cc:194] Registered new tserver with Master: d1a04dc5147d4759839dfd035fe2fc5d (127.24.136.2:45137)
I20260809 22:46:00.750074 29527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:43113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:46:00.774358 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:46055
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:01.120498 30127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:01.120829 30127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:01.120956 30127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:01.130435 30127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:01.130592 30127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:01.130770 30127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:01.141052 30127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:01.142570 30127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:01.144068 30127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:01.158320 30136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:01.158267 30137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:01.158293 30140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:01.159101 30127 server_base.cc:1034] running on GCE node
I20260809 22:46:01.159906 30127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:01.160965 30127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:01.162180 30127 hybrid_clock.cc:648] HybridClock initialized: now 1786315561162115 us; error 63 us; skew 500 ppm
I20260809 22:46:01.162513 30127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:01 dist-test-slave-pww0 krb5kdc[29451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315561, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:01.166669 30127 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:46:01.168612 30127 webserver.cc:460] Webserver started at http://127.24.136.3:44991/ using document root <none> and password file <none>
I20260809 22:46:01.169397 30127 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:01.169528 30127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:01.169886 30127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:01.173408 30127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "cdda22415c1a494fb5d8947878affcd4"
format_stamp: "Formatted at 2026-08-09 22:46:01 on dist-test-slave-pww0"
server_key: "e4e2ef56bd064151ed4fc81f1493ae65"
server_key_iv: "3d106595c57c9d18b1ae405506ed2be3"
server_key_version: "encryptionkey@0"
I20260809 22:46:01.174352 30127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "cdda22415c1a494fb5d8947878affcd4"
format_stamp: "Formatted at 2026-08-09 22:46:01 on dist-test-slave-pww0"
server_key: "e4e2ef56bd064151ed4fc81f1493ae65"
server_key_iv: "3d106595c57c9d18b1ae405506ed2be3"
server_key_version: "encryptionkey@0"
I20260809 22:46:01.180685 30127 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 22:46:01.185139 30148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:01.186890 30127 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:46:01.187096 30127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "cdda22415c1a494fb5d8947878affcd4"
format_stamp: "Formatted at 2026-08-09 22:46:01 on dist-test-slave-pww0"
server_key: "e4e2ef56bd064151ed4fc81f1493ae65"
server_key_iv: "3d106595c57c9d18b1ae405506ed2be3"
server_key_version: "encryptionkey@0"
I20260809 22:46:01.187316 30127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:01.200701 29875 heartbeater.cc:503] Master 127.24.136.62:46055 was elected leader, sending a full tablet report...
I20260809 22:46:01.207065 30127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:01.211818 30127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:01.212085 30127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:01.213126 30127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:01.214638 30127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:01.214715 30127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:01.214816 30127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:01.214861 30127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:01.248416 30127 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:39867
I20260809 22:46:01.248493 30344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:39867 every 8 connection(s)
I20260809 22:46:01.249779 30127 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:01.258388 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 30127
I20260809 22:46:01.258592 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:01.259146 25120 external_mini_cluster.cc:1428] Setting key cec8c57c972c6b7bc765e2353eb9844f
Aug 09 22:46:01 dist-test-slave-pww0 krb5kdc[29451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315561, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:01.271653 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:01.252423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:36367 (local address 127.24.136.62:46055)
0809 22:46:01.252890 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:01.252895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:01.253849 (+ 954us) server_negotiation.cc:407] Connection header received
0809 22:46:01.255884 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:01.255890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:01.255987 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:01.256084 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:01.257780 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:01.258890 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:01.260943 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:01.261427 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:01.261567 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:01.265380 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:01.265386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:01.265389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:01.265428 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:01.268249 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:01.269072 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:01.269076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:01.269078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:01.269209 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:01.269638 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:01.269641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:01.269642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:01.269930 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:01.270052 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:01.270956 (+ 904us) server_negotiation.cc:299] Negotiation successful
0809 22:46:01.271213 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":232,"threads_started":1}
I20260809 22:46:01.272548 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:01.252939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:46055 (local address 127.24.136.3:36367)
0809 22:46:01.253677 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:01.253726 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:01.255568 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:01.256366 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:01.256376 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:01.256934 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:01.257585 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:01.257605 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:01.259076 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:01.259082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:01.260791 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:01.260807 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:01.261706 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:01.261715 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:01.261946 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:01.262739 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:01.262771 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:01.265121 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:01.268427 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:01.268436 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:01.268454 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:01.268961 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:01.269310 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:01.269313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:01.269316 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:01.269537 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:01.270062 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:01.270069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:01.270461 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 22:46:01.270746 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 22:46:01.271246 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":282,"threads_started":1}
I20260809 22:46:01.274073 30345 heartbeater.cc:347] Connected to a master server at 127.24.136.62:46055
I20260809 22:46:01.274420 30345 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:01.275306 30345 heartbeater.cc:511] Master 127.24.136.62:46055 requested a full tablet report, sending...
I20260809 22:46:01.276955 29526 ts_manager.cc:194] Registered new tserver with Master: cdda22415c1a494fb5d8947878affcd4 (127.24.136.3:39867)
I20260809 22:46:01.277911 29526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:36367
I20260809 22:46:01.288203 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 22:46:01.328027 30348 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:01.325332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53250 (local address 127.24.136.62:46055)
0809 22:46:01.325564 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:01.325572 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:01.326037 (+ 465us) server_negotiation.cc:407] Connection header received
0809 22:46:01.326242 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:01.326247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:01.326321 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:01.326416 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:01.327829 (+ 1413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53250: BlockingRecv error: recv got EOF from 127.0.0.1:53250 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:46:01.331553 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29631
W20260809 22:46:01.334735 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29631: Not authorized: Client connection negotiation failed: client connection to 127.24.136.1:39205: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:46:01.334890 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 29631
I20260809 22:46:01.371793 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29882
W20260809 22:46:01.374615 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29882: Not authorized: Client connection negotiation failed: client connection to 127.24.136.2:45137: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:46:01.374816 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 29882
W20260809 22:46:01.374907 30367 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:01.372864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35322 (local address 127.24.136.2:45137)
0809 22:46:01.373362 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:01.373370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:01.373398 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:46:01.373463 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:01.373469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:01.373674 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:01.373836 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:01.374616 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35322: BlockingRecv error: recv got EOF from 127.0.0.1:35322 (error 108)
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":250,"threads_started":1}
I20260809 22:46:01.398047 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30127
W20260809 22:46:01.401389 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30127: Not authorized: Client connection negotiation failed: client connection to 127.24.136.3:39867: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:46:01.401531 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 30127
I20260809 22:46:01.433524 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29471
I20260809 22:46:01.434809 29573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:01.539984 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 29471
2026-08-09T22:46:01Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3963 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:01 dist-test-slave-pww0 krb5kdc[30377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:01 dist-test-slave-pww0 krb5kdc[30377](info): set up 2 sockets
Aug 09 22:46:01 dist-test-slave-pww0 krb5kdc[30377](info): commencing operation
krb5kdc: starting...
W20260809 22:46:02.803239 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.004s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:46:02 dist-test-slave-pww0 krb5kdc[30377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:46:03.033491 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38699
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:43985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:03.385623 30391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:03.385963 30391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:03.386087 30391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:03.393676 30391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:03.393826 30391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:03.393883 30391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:03.393966 30391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:03.394027 30391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:03.404608 30391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:43985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:03.406076 30391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:03.407555 30391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:03.420043 30400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:03.419957 30403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:03.420007 30401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:03.420853 30391 server_base.cc:1034] running on GCE node
I20260809 22:46:03.421705 30391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:03.423260 30391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:03.424489 30391 hybrid_clock.cc:648] HybridClock initialized: now 1786315563424437 us; error 49 us; skew 500 ppm
I20260809 22:46:03.424849 30391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:03 dist-test-slave-pww0 krb5kdc[30377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315563, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:03.430985 30391 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:46:03.432734 30391 webserver.cc:460] Webserver started at http://127.24.136.62:45079/ using document root <none> and password file <none>
I20260809 22:46:03.433468 30391 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:03.433579 30391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:03.433864 30391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:03.437109 30391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "93adfb7c5e3a44aaadced2acca6793ef"
format_stamp: "Formatted at 2026-08-09 22:46:03 on dist-test-slave-pww0"
server_key: "7afd5b1c86279663ee460095fdd8e564"
server_key_iv: "8df767a385b1d654f4ddfed14142cbee"
server_key_version: "encryptionkey@0"
I20260809 22:46:03.437949 30391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "93adfb7c5e3a44aaadced2acca6793ef"
format_stamp: "Formatted at 2026-08-09 22:46:03 on dist-test-slave-pww0"
server_key: "7afd5b1c86279663ee460095fdd8e564"
server_key_iv: "8df767a385b1d654f4ddfed14142cbee"
server_key_version: "encryptionkey@0"
I20260809 22:46:03.443687 30391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 22:46:03.447381 30418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:03.448873 30391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:03.449041 30391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "93adfb7c5e3a44aaadced2acca6793ef"
format_stamp: "Formatted at 2026-08-09 22:46:03 on dist-test-slave-pww0"
server_key: "7afd5b1c86279663ee460095fdd8e564"
server_key_iv: "8df767a385b1d654f4ddfed14142cbee"
server_key_version: "encryptionkey@0"
I20260809 22:46:03.449221 30391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:03.474162 30391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:03.478823 30391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:03.479092 30391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:03.498009 30391 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38699
I20260809 22:46:03.498044 30505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38699 every 8 connection(s)
I20260809 22:46:03.499506 30391 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:03.503571 30507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:03.509568 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 30391
I20260809 22:46:03.509842 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:03.510735 25120 external_mini_cluster.cc:1428] Setting key 50d77136ac0dbc49c46c2abfd7f2cf4e
I20260809 22:46:03.516403 30507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef: Bootstrap starting.
I20260809 22:46:03.520334 30507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:03.521731 30507 log.cc:824] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:03.525405 30507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef: No bootstrap required, opened a new log
Aug 09 22:46:03 dist-test-slave-pww0 krb5kdc[30377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:03.533905 30507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93adfb7c5e3a44aaadced2acca6793ef" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38699 } }
I20260809 22:46:03.534314 30507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:03.534425 30507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93adfb7c5e3a44aaadced2acca6793ef, State: Initialized, Role: FOLLOWER
I20260809 22:46:03.535071 30507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93adfb7c5e3a44aaadced2acca6793ef" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38699 } }
I20260809 22:46:03.535317 30507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:03.535476 30507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:03.535670 30507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:03.536072 30513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:03.513194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.24.136.62:38699)
0809 22:46:03.514159 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:03.514172 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:03.514219 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:46:03.516272 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:03.516301 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:03.516725 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:03.517081 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:03.519046 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:03.520505 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:03.522338 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:03.523405 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:03.523600 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:03.528332 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:03.528343 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:03.528364 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:03.528404 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:03.531623 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:03.532406 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:03.532414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:03.532421 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:03.532611 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:03.533010 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:03.533014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:03.533016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:03.533603 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:03.533893 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:03.534391 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 22:46:03.534878 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":415,"threads_started":1}
I20260809 22:46:03.538081 30507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93adfb7c5e3a44aaadced2acca6793ef" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38699 } }
I20260809 22:46:03.538643 30507 leader_election.cc:302] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93adfb7c5e3a44aaadced2acca6793ef; no voters:
I20260809 22:46:03.539352 30507 leader_election.cc:288] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:03.539511 30516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:03.540185 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:46:03.541736 30507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:03.547771 30516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [term 1 LEADER]: Becoming Leader. State: Replica: 93adfb7c5e3a44aaadced2acca6793ef, State: Running, Role: LEADER
I20260809 22:46:03.548529 30516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93adfb7c5e3a44aaadced2acca6793ef" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38699 } }
I20260809 22:46:03.552644 30517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93adfb7c5e3a44aaadced2acca6793ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93adfb7c5e3a44aaadced2acca6793ef" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38699 } } }
I20260809 22:46:03.553089 30517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:03.557456 30541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:03.559623 30522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93adfb7c5e3a44aaadced2acca6793ef. Latest consensus state: current_term: 1 leader_uuid: "93adfb7c5e3a44aaadced2acca6793ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93adfb7c5e3a44aaadced2acca6793ef" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38699 } } }
I20260809 22:46:03.560082 30522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:03.565294 30541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:03.586946 30541 catalog_manager.cc:1347] Generated new cluster ID: 301f2a4e5340471c9010d47aa30bcfb1
I20260809 22:46:03.587077 30541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:03.610905 30541 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:03.613269 30541 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:03.624716 30541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93adfb7c5e3a44aaadced2acca6793ef: Generated new TSK 0
I20260809 22:46:03.625893 30541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:46:03.630465 30513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:03.613003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33858 (local address 127.24.136.62:38699)
0809 22:46:03.613181 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:03.613189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:03.622009 (+ 8820us) server_negotiation.cc:407] Connection header received
0809 22:46:03.622243 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:03.622249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:03.622354 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:03.622461 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:03.623754 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:03.624905 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:03.626762 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:03.627524 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:03.627675 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:03.628761 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:03.628769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:03.628774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:03.629649 (+ 875us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:03.629733 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:03.629765 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:03.629887 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:03.629962 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 22:46:03.630082 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 22:46:03.638437 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30391
I20260809 22:46:03.639695 30498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:03.732336 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 30391
2026-08-09T22:46:03Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2182 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:03 dist-test-slave-pww0 krb5kdc[30574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:03 dist-test-slave-pww0 krb5kdc[30574](info): set up 2 sockets
Aug 09 22:46:03 dist-test-slave-pww0 krb5kdc[30574](info): commencing operation
krb5kdc: starting...
W20260809 22:46:04.980031 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:46:05 dist-test-slave-pww0 krb5kdc[30574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:46:05.211061 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45687
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:05.555534 30606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:05.555886 30606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:05.555991 30606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:05.563346 30606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:05.563494 30606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:05.563547 30606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:05.563606 30606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:05.563649 30606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:05.573912 30606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:05.575340 30606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:05.576752 30606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:05.588627 30615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:05.588590 30614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:05.588665 30618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:05.589526 30606 server_base.cc:1034] running on GCE node
I20260809 22:46:05.590399 30606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:05.591977 30606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:05.593183 30606 hybrid_clock.cc:648] HybridClock initialized: now 1786315565593132 us; error 36 us; skew 500 ppm
I20260809 22:46:05.593547 30606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:05 dist-test-slave-pww0 krb5kdc[30574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315565, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:05.596812 30606 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:46:05.598361 30606 webserver.cc:460] Webserver started at http://127.24.136.62:35229/ using document root <none> and password file <none>
I20260809 22:46:05.599066 30606 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:05.599189 30606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:05.599560 30606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:05.602743 30606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744"
format_stamp: "Formatted at 2026-08-09 22:46:05 on dist-test-slave-pww0"
server_key: "96e169205ba90407c0221e2faf0bb33a"
server_key_iv: "935fc2ebce59b3f61640e1d18d919880"
server_key_version: "encryptionkey@0"
I20260809 22:46:05.603654 30606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744"
format_stamp: "Formatted at 2026-08-09 22:46:05 on dist-test-slave-pww0"
server_key: "96e169205ba90407c0221e2faf0bb33a"
server_key_iv: "935fc2ebce59b3f61640e1d18d919880"
server_key_version: "encryptionkey@0"
I20260809 22:46:05.609428 30606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 22:46:05.613310 30628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:05.614755 30606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:46:05.614946 30606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744"
format_stamp: "Formatted at 2026-08-09 22:46:05 on dist-test-slave-pww0"
server_key: "96e169205ba90407c0221e2faf0bb33a"
server_key_iv: "935fc2ebce59b3f61640e1d18d919880"
server_key_version: "encryptionkey@0"
I20260809 22:46:05.615145 30606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:05.639906 30606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:05.644410 30606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:05.644711 30606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:05.663141 30606 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:45687
I20260809 22:46:05.663185 30712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:45687 every 8 connection(s)
I20260809 22:46:05.664608 30606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:05.668745 30713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:05.674835 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 30606
I20260809 22:46:05.675086 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:05.675956 25120 external_mini_cluster.cc:1428] Setting key bccb430a71832e2dea08340585219910
I20260809 22:46:05.680819 30713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744: Bootstrap starting.
I20260809 22:46:05.684095 30713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:05.685137 30713 log.cc:824] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:05.688690 30713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744: No bootstrap required, opened a new log
Aug 09 22:46:05 dist-test-slave-pww0 krb5kdc[30574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:05.698786 30713 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45687 } }
I20260809 22:46:05.699216 30713 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:05.699352 30713 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca7032d7fc6d4e09b8a56ebb34c1f744, State: Initialized, Role: FOLLOWER
I20260809 22:46:05.700021 30717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:05.678367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36574 (local address 127.24.136.62:45687)
0809 22:46:05.679215 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:05.679228 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:05.679273 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:46:05.681014 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:05.681040 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:05.681450 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:05.681832 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:05.683277 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:05.684652 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:05.686106 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:05.687130 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:05.687344 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:05.691943 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:05.691955 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:05.691972 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:05.692014 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:05.695276 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:05.696335 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:05.696343 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:05.696351 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:05.696559 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:05.697031 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:05.697036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:05.697039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:05.697650 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:05.697922 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:05.698382 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 22:46:05.698751 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":390,"threads_started":1}
I20260809 22:46:05.700065 30713 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45687 } }
I20260809 22:46:05.700369 30713 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:05.700546 30713 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:05.700783 30713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:05.703809 30713 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45687 } }
I20260809 22:46:05.704322 30713 leader_election.cc:302] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca7032d7fc6d4e09b8a56ebb34c1f744; no voters:
I20260809 22:46:05.704897 30713 leader_election.cc:288] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:05.705015 30719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:05.706130 30719 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [term 1 LEADER]: Becoming Leader. State: Replica: ca7032d7fc6d4e09b8a56ebb34c1f744, State: Running, Role: LEADER
I20260809 22:46:05.706665 30719 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45687 } }
I20260809 22:46:05.707370 30713 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:05.709674 30721 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45687 } } }
I20260809 22:46:05.710059 30721 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:05.709753 30723 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca7032d7fc6d4e09b8a56ebb34c1f744. Latest consensus state: current_term: 1 leader_uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca7032d7fc6d4e09b8a56ebb34c1f744" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45687 } } }
I20260809 22:46:05.715915 30723 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:05.717669 30733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:05.723726 30733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:05.733495 30733 catalog_manager.cc:1347] Generated new cluster ID: 386837872ca647cb993fbeb5caab46dc
I20260809 22:46:05.733642 30733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:05.751740 30733 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:05.753750 30733 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:05.766244 30733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca7032d7fc6d4e09b8a56ebb34c1f744: Generated new TSK 0
I20260809 22:46:05.767551 30733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:46:05.822712 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45687
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:06.176548 30765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:06.176873 30765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:06.176973 30765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:06.184319 30765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:06.184441 30765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:06.184602 30765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:06.194545 30765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:06.195979 30765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:06.197309 30765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:06.210692 30774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:06.210654 30780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:06.210721 30782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:06.211583 30765 server_base.cc:1034] running on GCE node
I20260809 22:46:06.212198 30765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:06.213187 30765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:06.214389 30765 hybrid_clock.cc:648] HybridClock initialized: now 1786315566214325 us; error 51 us; skew 500 ppm
I20260809 22:46:06.214744 30765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:06 dist-test-slave-pww0 krb5kdc[30574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315566, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:06.218017 30765 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:46:06.219643 30765 webserver.cc:460] Webserver started at http://127.24.136.1:41295/ using document root <none> and password file <none>
I20260809 22:46:06.220394 30765 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:06.220530 30765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:06.220798 30765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:06.223980 30765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "684e38c3f0f548928f449942bc4a7369"
format_stamp: "Formatted at 2026-08-09 22:46:06 on dist-test-slave-pww0"
server_key: "7fa7389f40333f4c582c6b6d796e51fb"
server_key_iv: "806aca233f073b53f3c84cdfa1a628a7"
server_key_version: "encryptionkey@0"
I20260809 22:46:06.224944 30765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "684e38c3f0f548928f449942bc4a7369"
format_stamp: "Formatted at 2026-08-09 22:46:06 on dist-test-slave-pww0"
server_key: "7fa7389f40333f4c582c6b6d796e51fb"
server_key_iv: "806aca233f073b53f3c84cdfa1a628a7"
server_key_version: "encryptionkey@0"
I20260809 22:46:06.230443 30765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 22:46:06.234094 30794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:06.235492 30765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:46:06.235666 30765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "684e38c3f0f548928f449942bc4a7369"
format_stamp: "Formatted at 2026-08-09 22:46:06 on dist-test-slave-pww0"
server_key: "7fa7389f40333f4c582c6b6d796e51fb"
server_key_iv: "806aca233f073b53f3c84cdfa1a628a7"
server_key_version: "encryptionkey@0"
I20260809 22:46:06.235862 30765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:06.263360 30765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:06.267925 30765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:06.268224 30765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:06.269260 30765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:06.270710 30765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:06.270794 30765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:06.270910 30765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:06.270964 30765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:06.302335 30765 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:37465
I20260809 22:46:06.302384 30993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:37465 every 8 connection(s)
I20260809 22:46:06.303759 30765 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:06.306638 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 30765
I20260809 22:46:06.306874 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:06.307654 25120 external_mini_cluster.cc:1428] Setting key 558d12b56a1915667206414753447bd1
Aug 09 22:46:06 dist-test-slave-pww0 krb5kdc[30574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315566, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:06.327042 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:06.306483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:55679 (local address 127.24.136.62:45687)
0809 22:46:06.306996 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:06.307003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:06.308114 (+ 1111us) server_negotiation.cc:407] Connection header received
0809 22:46:06.310342 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:06.310346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:06.310412 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:06.310505 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:06.312621 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:06.313331 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:06.315573 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:06.316000 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:06.316102 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:06.320232 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:06.320240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:06.320245 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:06.320290 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:06.323653 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:06.324501 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:06.324506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:06.324509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:06.324673 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:06.325091 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:06.325096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:06.325100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:06.325499 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:06.325646 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:06.326393 (+ 747us) server_negotiation.cc:299] Negotiation successful
0809 22:46:06.326658 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":254,"threads_started":1}
I20260809 22:46:06.327946 30998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:06.307017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45687 (local address 127.24.136.1:55679)
0809 22:46:06.307922 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:06.307973 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:06.309995 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:06.310803 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:06.310815 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:06.311600 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:06.312445 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:06.312470 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:06.313511 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:06.313516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:06.315358 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:06.315379 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:06.316239 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:06.316245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:06.316535 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:06.317303 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:06.317334 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:06.319920 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:06.323832 (+ 3912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:06.323841 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:06.323857 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:06.324349 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:06.324771 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:06.324774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:06.324777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:06.324978 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:06.325631 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:06.325638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:06.326000 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 22:46:06.326247 (+ 247us) client_negotiation.cc:240] Negotiation successful
0809 22:46:06.326682 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":348,"threads_started":1}
I20260809 22:46:06.329644 30994 heartbeater.cc:347] Connected to a master server at 127.24.136.62:45687
I20260809 22:46:06.330025 30994 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:06.331050 30994 heartbeater.cc:511] Master 127.24.136.62:45687 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
I20260809 22:46:06.333698 30656 ts_manager.cc:194] Registered new tserver with Master: 684e38c3f0f548928f449942bc4a7369 (127.24.136.1:37465)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
I20260809 22:46:06.335410 30656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:55679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:46:06.363142 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45687
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:06.712263 31003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:06.712605 31003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:06.712746 31003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:06.720949 31003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:06.721114 31003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:06.721287 31003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:06.732151 31003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:06.733559 31003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:06.734953 31003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:06.749090 31011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:06.749053 31013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:06.749018 31015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:06.750174 31003 server_base.cc:1034] running on GCE node
I20260809 22:46:06.750871 31003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:06.751983 31003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:06.753229 31003 hybrid_clock.cc:648] HybridClock initialized: now 1786315566753169 us; error 50 us; skew 500 ppm
I20260809 22:46:06.753628 31003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:06 dist-test-slave-pww0 krb5kdc[30574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315566, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:06.757067 31003 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:46:06.758800 31003 webserver.cc:460] Webserver started at http://127.24.136.2:35443/ using document root <none> and password file <none>
I20260809 22:46:06.759615 31003 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:06.759773 31003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:06.760104 31003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:06.763273 31003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "d36ca5cd5f9047b1952e126173bcbe8a"
format_stamp: "Formatted at 2026-08-09 22:46:06 on dist-test-slave-pww0"
server_key: "8136fbeebc9d33beb9412610a0cfbf15"
server_key_iv: "29614e7ddfde89dd4bab04fb30ed272b"
server_key_version: "encryptionkey@0"
I20260809 22:46:06.764180 31003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "d36ca5cd5f9047b1952e126173bcbe8a"
format_stamp: "Formatted at 2026-08-09 22:46:06 on dist-test-slave-pww0"
server_key: "8136fbeebc9d33beb9412610a0cfbf15"
server_key_iv: "29614e7ddfde89dd4bab04fb30ed272b"
server_key_version: "encryptionkey@0"
I20260809 22:46:06.770045 31003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 22:46:06.774276 31029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:06.776036 31003 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 22:46:06.776245 31003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "d36ca5cd5f9047b1952e126173bcbe8a"
format_stamp: "Formatted at 2026-08-09 22:46:06 on dist-test-slave-pww0"
server_key: "8136fbeebc9d33beb9412610a0cfbf15"
server_key_iv: "29614e7ddfde89dd4bab04fb30ed272b"
server_key_version: "encryptionkey@0"
I20260809 22:46:06.776484 31003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:06.802599 31003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:06.807168 31003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:06.807526 31003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:06.808672 31003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:06.810170 31003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:06.810252 31003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:06.810401 31003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:06.810456 31003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:06.843158 31003 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:32823
I20260809 22:46:06.843225 31232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:32823 every 8 connection(s)
I20260809 22:46:06.844563 31003 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:06.847842 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31003
I20260809 22:46:06.848055 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:06.848623 25120 external_mini_cluster.cc:1428] Setting key ab1cd1c496b71994936b0c3a8ae5953f
Aug 09 22:46:06 dist-test-slave-pww0 krb5kdc[30574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315566, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:06.867117 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:06.847537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:48609 (local address 127.24.136.62:45687)
0809 22:46:06.848081 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:06.848088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:06.849234 (+ 1146us) server_negotiation.cc:407] Connection header received
0809 22:46:06.851519 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:06.851525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:06.851615 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:06.851730 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:06.853456 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:06.854422 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:06.856279 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:06.856847 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:06.856973 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:06.861164 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:06.861169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:06.861173 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:06.861209 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:06.863783 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:06.864588 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:06.864593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:06.864595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:06.864731 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:06.865178 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:06.865181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:06.865183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:06.865459 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:06.865606 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:06.866444 (+ 838us) server_negotiation.cc:299] Negotiation successful
0809 22:46:06.866704 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":261,"threads_started":1}
I20260809 22:46:06.867966 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:06.847995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45687 (local address 127.24.136.2:48609)
0809 22:46:06.848982 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:06.849067 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:06.851181 (+ 2114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:06.851959 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:06.851968 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:06.852573 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:06.853246 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:06.853269 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:06.854614 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:06.854619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:06.856091 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:06.856108 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:06.857117 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:06.857123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:06.857354 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:06.858102 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:06.858129 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:06.860889 (+ 2760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:06.863941 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:06.863953 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:06.863971 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:06.864441 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:06.864844 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:06.864849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:06.864852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:06.865051 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:06.865596 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:06.865605 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:06.865969 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 22:46:06.866275 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 22:46:06.866729 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":355,"threads_started":1}
I20260809 22:46:06.869446 31233 heartbeater.cc:347] Connected to a master server at 127.24.136.62:45687
I20260809 22:46:06.869760 31233 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
I20260809 22:46:06.870527 31233 heartbeater.cc:511] Master 127.24.136.62:45687 requested a full tablet report, sending...
I20260809 22:46:06.872212 30656 ts_manager.cc:194] Registered new tserver with Master: d36ca5cd5f9047b1952e126173bcbe8a (127.24.136.2:32823)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
I20260809 22:46:06.873250 30656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:48609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:46:06.898856 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45687
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:07.265720 31241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:07.266080 31241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:07.266222 31241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:07.273908 31241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:07.274052 31241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:07.274211 31241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:07.284405 31241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:45687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:07.285815 31241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:07.287191 31241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:07.301370 31248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:07.301398 31247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:07.302001 31254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:07.304431 31241 server_base.cc:1034] running on GCE node
I20260809 22:46:07.305145 31241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:07.306222 31241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:07.307482 31241 hybrid_clock.cc:648] HybridClock initialized: now 1786315567307390 us; error 63 us; skew 500 ppm
I20260809 22:46:07.307852 31241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[30574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315567, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:07.311203 31241 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:46:07.312958 31241 webserver.cc:460] Webserver started at http://127.24.136.3:45835/ using document root <none> and password file <none>
I20260809 22:46:07.313745 31241 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:07.313865 31241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:07.314184 31241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:07.317370 31241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "01022af70d8f4a87b5a240f7acf951ae"
format_stamp: "Formatted at 2026-08-09 22:46:07 on dist-test-slave-pww0"
server_key: "5cafc393744a024a535dc87dbe00b951"
server_key_iv: "ce48f7a02140c939566ebbb7331bb6da"
server_key_version: "encryptionkey@0"
I20260809 22:46:07.318301 31241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "01022af70d8f4a87b5a240f7acf951ae"
format_stamp: "Formatted at 2026-08-09 22:46:07 on dist-test-slave-pww0"
server_key: "5cafc393744a024a535dc87dbe00b951"
server_key_iv: "ce48f7a02140c939566ebbb7331bb6da"
server_key_version: "encryptionkey@0"
I20260809 22:46:07.324548 31241 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 22:46:07.328438 31266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:07.330005 31241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:07.330206 31241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "01022af70d8f4a87b5a240f7acf951ae"
format_stamp: "Formatted at 2026-08-09 22:46:07 on dist-test-slave-pww0"
server_key: "5cafc393744a024a535dc87dbe00b951"
server_key_iv: "ce48f7a02140c939566ebbb7331bb6da"
server_key_version: "encryptionkey@0"
I20260809 22:46:07.330452 31241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:07.339288 30994 heartbeater.cc:503] Master 127.24.136.62:45687 was elected leader, sending a full tablet report...
I20260809 22:46:07.360754 31241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:07.365374 31241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:07.365777 31241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:07.367041 31241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:07.368774 31241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:07.368872 31241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:07.369006 31241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:07.369060 31241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:07.402204 31241 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:40909
I20260809 22:46:07.402251 31462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:40909 every 8 connection(s)
I20260809 22:46:07.403630 31241 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:07.413800 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31241
I20260809 22:46:07.413972 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:07.414466 25120 external_mini_cluster.cc:1428] Setting key 7685e9b95e6028607977e257942a937b
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[30574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315567, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:07.426198 31468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:07.406444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:37313 (local address 127.24.136.62:45687)
0809 22:46:07.406932 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:07.406937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:07.408271 (+ 1334us) server_negotiation.cc:407] Connection header received
0809 22:46:07.410363 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:07.410368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:07.410466 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:07.410579 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:07.412385 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.413176 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.414983 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.415626 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.415752 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:07.419892 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:07.419898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:07.419901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:07.420043 (+ 142us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:07.422862 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.423893 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.423899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.423901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.424049 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.424493 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.424497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.424499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.424810 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:07.424943 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:07.425703 (+ 760us) server_negotiation.cc:299] Negotiation successful
0809 22:46:07.425897 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":256,"threads_started":1}
I20260809 22:46:07.427196 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:07.407062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45687 (local address 127.24.136.3:37313)
0809 22:46:07.408068 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:07.408121 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:07.410022 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:07.410876 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:07.410884 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:07.411543 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:07.412187 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:07.412208 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.413343 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.413347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:07.414808 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:07.414824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.415883 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.415890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:07.416152 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:07.416861 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:07.416887 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:07.419649 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:07.423067 (+ 3418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:07.423077 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:07.423093 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:07.423737 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:07.424152 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:07.424155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:07.424157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:07.424367 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:07.424925 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:07.424932 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:07.425318 (+ 386us) client_negotiation.cc:769] Sending connection context
0809 22:46:07.425556 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 22:46:07.425999 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":399,"threads_started":1}
I20260809 22:46:07.428866 31463 heartbeater.cc:347] Connected to a master server at 127.24.136.62:45687
I20260809 22:46:07.429217 31463 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:07.430102 31463 heartbeater.cc:511] Master 127.24.136.62:45687 requested a full tablet report, sending...
I20260809 22:46:07.431671 30656 ts_manager.cc:194] Registered new tserver with Master: 01022af70d8f4a87b5a240f7acf951ae (127.24.136.3:40909)
I20260809 22:46:07.432482 30656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:37313
I20260809 22:46:07.442981 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[30574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315567, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[30574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315567, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:07.499122 31468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:07.484281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.24.136.62:45687)
0809 22:46:07.484454 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:07.484461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:07.484931 (+ 470us) server_negotiation.cc:407] Connection header received
0809 22:46:07.485112 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:07.485116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:07.485184 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:07.485269 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:07.486652 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.487406 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.488716 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.489131 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.489245 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:07.493391 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:07.493399 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:07.493404 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:07.493455 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:07.496345 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.497070 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.497075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.497077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.497217 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.497607 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.497610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.497613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.497964 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:07.498205 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:07.498558 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 22:46:07.498800 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 22:46:07.499687 30656 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36588
I20260809 22:46:07.501315 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30765
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[30574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315567, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:46:07.519711 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:07.502623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48872 (local address 127.24.136.1:37465)
0809 22:46:07.503258 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:07.503266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:07.503292 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:46:07.503352 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:07.503359 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:07.503649 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:07.503791 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:07.505090 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.506162 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.507576 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.508354 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.508461 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:07.513002 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:07.513018 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:07.513034 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:07.513088 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:07.516244 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.516991 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.516999 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.517002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.517178 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.517658 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.517662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.517664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.518162 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:07.518425 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:07.518836 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 22:46:07.519304 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":328,"threads_started":1}
W20260809 22:46:07.521006 30987 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48872
W20260809 22:46:07.522136 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30765: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:46:07.522387 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 30765
I20260809 22:46:07.554587 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31003
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[30574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315567, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:46:07.571599 31488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:07.555589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.24.136.2:32823)
0809 22:46:07.556182 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:07.556189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:07.556217 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:46:07.556282 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:07.556289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:07.556511 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:07.556644 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:07.557942 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.559015 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.560451 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.561085 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.561212 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:07.565560 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:07.565577 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:07.565594 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:07.565636 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:07.568621 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.569418 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.569426 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.569428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.569570 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.569932 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.569936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.569938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.570345 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:07.570606 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:07.570943 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 22:46:07.571176 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":290,"threads_started":1}
W20260809 22:46:07.572889 31225 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51202
W20260809 22:46:07.574071 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31003: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:46:07.574245 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31003
I20260809 22:46:07.600716 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31241
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[30574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315567, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:46:07.617877 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:07.601674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.24.136.3:40909)
0809 22:46:07.602350 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:07.602357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:07.602389 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:46:07.602455 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:07.602462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:07.602679 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:07.602818 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:07.604423 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.605534 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.606839 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:07.607521 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:07.607628 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:07.612051 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:07.612064 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:07.612081 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:07.612126 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:07.614943 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.615826 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.615835 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.615837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.615969 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:07.616395 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:07.616398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:07.616400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:07.616770 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:07.617019 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:07.617348 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 22:46:07.617578 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":329,"threads_started":1}
W20260809 22:46:07.619383 31455 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45104
W20260809 22:46:07.620818 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31241: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:46:07.621011 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31241
I20260809 22:46:07.643963 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30606
I20260809 22:46:07.645300 30708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:07.750269 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 30606
2026-08-09T22:46:07Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4025 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[31498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[31498](info): set up 2 sockets
Aug 09 22:46:07 dist-test-slave-pww0 krb5kdc[31498](info): commencing operation
krb5kdc: starting...
W20260809 22:46:08.996658 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:46:09 dist-test-slave-pww0 krb5kdc[31498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:46:09.238710 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:40205
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:40205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:09.587858 31519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:09.588176 31519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:09.588315 31519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:09.595827 31519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:09.595960 31519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:09.596012 31519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:09.596092 31519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:09.596153 31519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:09.606403 31519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:40205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:40205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:09.607841 31519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:09.609237 31519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:09.621426 31535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:09.621405 31536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:09.621470 31540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:09.622679 31519 server_base.cc:1034] running on GCE node
I20260809 22:46:09.623540 31519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:09.625135 31519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:09.626376 31519 hybrid_clock.cc:648] HybridClock initialized: now 1786315569626320 us; error 50 us; skew 500 ppm
I20260809 22:46:09.626791 31519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:09 dist-test-slave-pww0 krb5kdc[31498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315569, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:09.633864 31519 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:46:09.635809 31519 webserver.cc:460] Webserver started at http://127.24.136.62:37321/ using document root <none> and password file <none>
I20260809 22:46:09.636636 31519 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:09.636786 31519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:09.637156 31519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:09.640699 31519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "77b9fab556494ea88b6dda8f0fd0ec3b"
format_stamp: "Formatted at 2026-08-09 22:46:09 on dist-test-slave-pww0"
server_key: "4546cbb4fadb76e5712f4687138ff1fe"
server_key_iv: "8b925aeb018254590b4d3e6d608f3540"
server_key_version: "encryptionkey@0"
I20260809 22:46:09.641645 31519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "77b9fab556494ea88b6dda8f0fd0ec3b"
format_stamp: "Formatted at 2026-08-09 22:46:09 on dist-test-slave-pww0"
server_key: "4546cbb4fadb76e5712f4687138ff1fe"
server_key_iv: "8b925aeb018254590b4d3e6d608f3540"
server_key_version: "encryptionkey@0"
I20260809 22:46:09.647753 31519 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260809 22:46:09.651769 31552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:09.653353 31519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:09.653554 31519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "77b9fab556494ea88b6dda8f0fd0ec3b"
format_stamp: "Formatted at 2026-08-09 22:46:09 on dist-test-slave-pww0"
server_key: "4546cbb4fadb76e5712f4687138ff1fe"
server_key_iv: "8b925aeb018254590b4d3e6d608f3540"
server_key_version: "encryptionkey@0"
I20260809 22:46:09.653789 31519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:09.689450 31519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:09.694283 31519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:09.694585 31519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:09.712862 31519 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:40205
I20260809 22:46:09.712894 31638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:40205 every 8 connection(s)
I20260809 22:46:09.714350 31519 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:09.718291 31639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:09.723249 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31519
I20260809 22:46:09.723541 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:09.724375 25120 external_mini_cluster.cc:1428] Setting key 6f6ce19ed0f15ccf5b056cad39a5dbd4
I20260809 22:46:09.730569 31639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b: Bootstrap starting.
I20260809 22:46:09.734549 31639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:09.736011 31639 log.cc:824] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:09.739264 31639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b: No bootstrap required, opened a new log
Aug 09 22:46:09 dist-test-slave-pww0 krb5kdc[31498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:09.749917 31639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b9fab556494ea88b6dda8f0fd0ec3b" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 40205 } }
I20260809 22:46:09.750439 31639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:09.750564 31639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b9fab556494ea88b6dda8f0fd0ec3b, State: Initialized, Role: FOLLOWER
I20260809 22:46:09.750811 31643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:09.726654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.24.136.62:40205)
0809 22:46:09.727572 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:09.727589 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:09.727640 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:46:09.729466 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:09.729494 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:09.730010 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:09.730408 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:09.732523 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:09.733933 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:09.735840 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:09.736812 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:09.737042 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:09.740933 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:09.740944 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:09.740966 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:09.741026 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:09.745481 (+ 4455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:09.746378 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:09.746390 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:09.746406 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:09.746668 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:09.747096 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:09.747105 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:09.747109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:09.748032 (+ 923us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:09.748372 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:09.748785 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 22:46:09.749296 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":394,"threads_started":1}
I20260809 22:46:09.751310 31639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b9fab556494ea88b6dda8f0fd0ec3b" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 40205 } }
I20260809 22:46:09.751653 31639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:09.751804 31639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:09.751976 31639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:09.754599 31639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b9fab556494ea88b6dda8f0fd0ec3b" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 40205 } }
I20260809 22:46:09.755084 31639 leader_election.cc:302] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77b9fab556494ea88b6dda8f0fd0ec3b; no voters:
I20260809 22:46:09.755822 31639 leader_election.cc:288] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:09.755951 31645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:09.756911 31645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [term 1 LEADER]: Becoming Leader. State: Replica: 77b9fab556494ea88b6dda8f0fd0ec3b, State: Running, Role: LEADER
I20260809 22:46:09.757537 31645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b9fab556494ea88b6dda8f0fd0ec3b" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 40205 } }
I20260809 22:46:09.758280 31639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:09.760980 31646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77b9fab556494ea88b6dda8f0fd0ec3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b9fab556494ea88b6dda8f0fd0ec3b" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 40205 } } }
I20260809 22:46:09.761415 31646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:09.761425 31649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77b9fab556494ea88b6dda8f0fd0ec3b. Latest consensus state: current_term: 1 leader_uuid: "77b9fab556494ea88b6dda8f0fd0ec3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b9fab556494ea88b6dda8f0fd0ec3b" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 40205 } } }
I20260809 22:46:09.762197 31649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:09.765064 31661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:09.770704 31661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:09.780104 31661 catalog_manager.cc:1347] Generated new cluster ID: b9ec54328f984d33b8d9e9d9a9396194
I20260809 22:46:09.780242 31661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:09.798218 31661 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:09.799886 31661 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:09.816075 31661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77b9fab556494ea88b6dda8f0fd0ec3b: Generated new TSK 0
I20260809 22:46:09.816967 31661 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:46:09.877239 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:40205
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:10.226011 31680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:10.226333 31680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:10.226434 31680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:10.234153 31680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:10.234289 31680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:10.234408 31680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:10.244738 31680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:40205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:10.246070 31680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:10.247411 31680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:10.261476 31687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:10.261513 31690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:10.261440 31695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:10.262161 31680 server_base.cc:1034] running on GCE node
I20260809 22:46:10.262812 31680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:10.263852 31680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:10.265066 31680 hybrid_clock.cc:648] HybridClock initialized: now 1786315570265006 us; error 46 us; skew 500 ppm
I20260809 22:46:10.265419 31680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:10 dist-test-slave-pww0 krb5kdc[31498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:10.268775 31680 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:46:10.270431 31680 webserver.cc:460] Webserver started at http://127.24.136.1:45673/ using document root <none> and password file <none>
I20260809 22:46:10.271206 31680 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:10.271382 31680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:10.271786 31680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:10.275058 31680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "d662e193ff754e85923d77e3a50ee300"
format_stamp: "Formatted at 2026-08-09 22:46:10 on dist-test-slave-pww0"
server_key: "19ecd222517db681a652ef0bbc970921"
server_key_iv: "cb3e1f7b8bda93d790e0e2a08f6ddcb1"
server_key_version: "encryptionkey@0"
I20260809 22:46:10.275993 31680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "d662e193ff754e85923d77e3a50ee300"
format_stamp: "Formatted at 2026-08-09 22:46:10 on dist-test-slave-pww0"
server_key: "19ecd222517db681a652ef0bbc970921"
server_key_iv: "cb3e1f7b8bda93d790e0e2a08f6ddcb1"
server_key_version: "encryptionkey@0"
I20260809 22:46:10.281745 31680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:46:10.285851 31709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:10.287407 31680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:46:10.287637 31680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "d662e193ff754e85923d77e3a50ee300"
format_stamp: "Formatted at 2026-08-09 22:46:10 on dist-test-slave-pww0"
server_key: "19ecd222517db681a652ef0bbc970921"
server_key_iv: "cb3e1f7b8bda93d790e0e2a08f6ddcb1"
server_key_version: "encryptionkey@0"
I20260809 22:46:10.287851 31680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:10.309134 31680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:10.313553 31680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:10.313845 31680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:10.314967 31680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:10.316474 31680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:10.316555 31680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:10.316679 31680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:10.316730 31680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:10.347896 31680 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:42631
I20260809 22:46:10.347946 31902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:42631 every 8 connection(s)
I20260809 22:46:10.349254 31680 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:10.350719 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31680
I20260809 22:46:10.350978 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:10.351789 25120 external_mini_cluster.cc:1428] Setting key 33c6f8087b579cab8c78c52196bd230b
Aug 09 22:46:10 dist-test-slave-pww0 krb5kdc[31498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:10.372184 31905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:10.352248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:35025 (local address 127.24.136.62:40205)
0809 22:46:10.352739 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:10.352743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:10.353963 (+ 1220us) server_negotiation.cc:407] Connection header received
0809 22:46:10.355983 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:10.355989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:10.356078 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:10.356193 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:10.357946 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:10.358978 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:10.360764 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:10.361300 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:10.361420 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:10.365387 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:10.365394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:10.365398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:10.365445 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:10.368699 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:10.369596 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:10.369602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:10.369605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:10.369756 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:10.370197 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:10.370202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:10.370205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:10.370588 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:10.370731 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:10.371540 (+ 809us) server_negotiation.cc:299] Negotiation successful
0809 22:46:10.371815 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":230,"threads_started":1}
I20260809 22:46:10.373246 31906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:10.352931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:40205 (local address 127.24.136.1:35025)
0809 22:46:10.353788 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:10.353844 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:10.355676 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:10.356411 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:10.356419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:10.357037 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:10.357719 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:10.357739 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:10.359144 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:10.359148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:10.360605 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:10.360620 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:10.361519 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:10.361524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:10.361737 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:10.362502 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:10.362531 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:10.365079 (+ 2548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:10.368894 (+ 3815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:10.368904 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:10.368921 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:10.369439 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:10.369855 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:10.369858 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:10.369861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:10.370065 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:10.370707 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:10.370714 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:10.371070 (+ 356us) client_negotiation.cc:769] Sending connection context
0809 22:46:10.371340 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 22:46:10.371964 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":338,"threads_started":1}
I20260809 22:46:10.374989 31903 heartbeater.cc:347] Connected to a master server at 127.24.136.62:40205
I20260809 22:46:10.375353 31903 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
I20260809 22:46:10.376251 31903 heartbeater.cc:511] Master 127.24.136.62:40205 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
I20260809 22:46:10.378664 31574 ts_manager.cc:194] Registered new tserver with Master: d662e193ff754e85923d77e3a50ee300 (127.24.136.1:42631)
I20260809 22:46:10.380277 31574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:35025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:46:10.404918 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:40205
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:10.758307 31914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:10.758633 31914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:10.758776 31914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:10.766456 31914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:10.766593 31914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:10.766741 31914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:10.777253 31914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:40205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:10.778654 31914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:10.780128 31914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:10.793823 31920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:10.793897 31926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:10.793864 31923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:10.794661 31914 server_base.cc:1034] running on GCE node
I20260809 22:46:10.795333 31914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:10.796404 31914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:10.797608 31914 hybrid_clock.cc:648] HybridClock initialized: now 1786315570797539 us; error 68 us; skew 500 ppm
I20260809 22:46:10.797940 31914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:10 dist-test-slave-pww0 krb5kdc[31498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:10.801316 31914 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:46:10.802992 31914 webserver.cc:460] Webserver started at http://127.24.136.2:45579/ using document root <none> and password file <none>
I20260809 22:46:10.803857 31914 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:10.803995 31914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:10.804338 31914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:10.807607 31914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "1a2d7c25a7fd402ca02da1b83b38905c"
format_stamp: "Formatted at 2026-08-09 22:46:10 on dist-test-slave-pww0"
server_key: "d9a46abe10cef5f62bc8bab0a5df8720"
server_key_iv: "534459505ca1e5131c4f5dd210362287"
server_key_version: "encryptionkey@0"
I20260809 22:46:10.808473 31914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "1a2d7c25a7fd402ca02da1b83b38905c"
format_stamp: "Formatted at 2026-08-09 22:46:10 on dist-test-slave-pww0"
server_key: "d9a46abe10cef5f62bc8bab0a5df8720"
server_key_iv: "534459505ca1e5131c4f5dd210362287"
server_key_version: "encryptionkey@0"
I20260809 22:46:10.814165 31914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:46:10.817994 31935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:10.819528 31914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:46:10.819697 31914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "1a2d7c25a7fd402ca02da1b83b38905c"
format_stamp: "Formatted at 2026-08-09 22:46:10 on dist-test-slave-pww0"
server_key: "d9a46abe10cef5f62bc8bab0a5df8720"
server_key_iv: "534459505ca1e5131c4f5dd210362287"
server_key_version: "encryptionkey@0"
I20260809 22:46:10.819891 31914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:10.857003 31914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:10.861985 31914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:10.862293 31914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:10.863349 31914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:10.864847 31914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:10.864926 31914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:10.865052 31914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:10.865103 31914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:10.897617 31914 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:45013
I20260809 22:46:10.897696 32120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:45013 every 8 connection(s)
I20260809 22:46:10.899021 31914 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:10.899503 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31914
I20260809 22:46:10.899716 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:10.900276 25120 external_mini_cluster.cc:1428] Setting key f38e40943ae4dfdc01e2909a8ff5ad0a
Aug 09 22:46:10 dist-test-slave-pww0 krb5kdc[31498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315570, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:10.923532 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:10.901795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:48891 (local address 127.24.136.62:40205)
0809 22:46:10.902284 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:10.902290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:10.903518 (+ 1228us) server_negotiation.cc:407] Connection header received
0809 22:46:10.906195 (+ 2677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:10.906201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:10.906292 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:10.906408 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:10.908684 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:10.909628 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:10.911915 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:10.912366 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:10.912492 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:10.916662 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:10.916670 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:10.916676 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:10.916729 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:10.919666 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:10.920771 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:10.920776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:10.920779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:10.920929 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:10.921437 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:10.921444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:10.921447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:10.921857 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:10.922027 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:10.922812 (+ 785us) server_negotiation.cc:299] Negotiation successful
0809 22:46:10.923065 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":349,"threads_started":1}
I20260809 22:46:10.924497 32124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:10.902432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:40205 (local address 127.24.136.2:48891)
0809 22:46:10.903278 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:10.903373 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:10.905785 (+ 2412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:10.906732 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:10.906746 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:10.907593 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:10.908477 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:10.908506 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:10.909819 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:10.909824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:10.911737 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:10.911756 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:10.912622 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:10.912630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:10.912881 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:10.913561 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:10.913593 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:10.916235 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:10.919856 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:10.919867 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:10.919891 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:10.920602 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:10.921054 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:10.921058 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:10.921060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:10.921301 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:10.921990 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:10.921996 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:10.922377 (+ 381us) client_negotiation.cc:769] Sending connection context
0809 22:46:10.922620 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 22:46:10.923092 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":307,"threads_started":1}
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
I20260809 22:46:10.926266 32121 heartbeater.cc:347] Connected to a master server at 127.24.136.62:40205
I20260809 22:46:10.926638 32121 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:10.927640 32121 heartbeater.cc:511] Master 127.24.136.62:40205 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
I20260809 22:46:10.929411 31574 ts_manager.cc:194] Registered new tserver with Master: 1a2d7c25a7fd402ca02da1b83b38905c (127.24.136.2:45013)
I20260809 22:46:10.930320 31574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:48891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:46:10.956015 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:40205
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:11.309202 32128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:11.309523 32128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:11.309667 32128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:11.317297 32128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:11.317461 32128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:11.317615 32128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:11.328486 32128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:40205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:11.330054 32128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:11.331810 32128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:11.345568 32137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:11.345604 32140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:11.345520 32138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:11.346416 32128 server_base.cc:1034] running on GCE node
I20260809 22:46:11.347122 32128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:11.348203 32128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:11.349467 32128 hybrid_clock.cc:648] HybridClock initialized: now 1786315571349387 us; error 67 us; skew 500 ppm
I20260809 22:46:11.349839 32128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:11 dist-test-slave-pww0 krb5kdc[31498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315571, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:11.353437 32128 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:46:11.355006 32128 webserver.cc:460] Webserver started at http://127.24.136.3:32817/ using document root <none> and password file <none>
I20260809 22:46:11.355782 32128 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:11.355901 32128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:11.356227 32128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:11.359544 32128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "67032a6f49254488bd119fdf7b648928"
format_stamp: "Formatted at 2026-08-09 22:46:11 on dist-test-slave-pww0"
server_key: "6b52602f61dbc37809401d5da0167d44"
server_key_iv: "cd4cc987ab462d9c33da61109e9671c8"
server_key_version: "encryptionkey@0"
I20260809 22:46:11.360392 32128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "67032a6f49254488bd119fdf7b648928"
format_stamp: "Formatted at 2026-08-09 22:46:11 on dist-test-slave-pww0"
server_key: "6b52602f61dbc37809401d5da0167d44"
server_key_iv: "cd4cc987ab462d9c33da61109e9671c8"
server_key_version: "encryptionkey@0"
I20260809 22:46:11.366135 32128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:46:11.370200 32154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:11.371829 32128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:46:11.372015 32128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "67032a6f49254488bd119fdf7b648928"
format_stamp: "Formatted at 2026-08-09 22:46:11 on dist-test-slave-pww0"
server_key: "6b52602f61dbc37809401d5da0167d44"
server_key_iv: "cd4cc987ab462d9c33da61109e9671c8"
server_key_version: "encryptionkey@0"
I20260809 22:46:11.372242 32128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:11.383695 31903 heartbeater.cc:503] Master 127.24.136.62:40205 was elected leader, sending a full tablet report...
I20260809 22:46:11.383752 32128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:11.388514 32128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:11.388830 32128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:11.389860 32128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:11.391440 32128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:11.391566 32128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 22:46:11.391696 32128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:11.391759 32128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:11.421860 32128 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:40883
I20260809 22:46:11.421909 32336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:40883 every 8 connection(s)
I20260809 22:46:11.423247 32128 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:11.431438 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 32128
I20260809 22:46:11.431664 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:11.432096 25120 external_mini_cluster.cc:1428] Setting key 41784a054bf1e952236a37778a3c576e
Aug 09 22:46:11 dist-test-slave-pww0 krb5kdc[31498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315571, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:11.444626 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:11.426224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:36751 (local address 127.24.136.62:40205)
0809 22:46:11.426708 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:11.426713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:11.427827 (+ 1114us) server_negotiation.cc:407] Connection header received
0809 22:46:11.429771 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:11.429778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:11.429867 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:11.429968 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:11.431704 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.432735 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.434606 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.435086 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.435225 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:11.438917 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:11.438923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:11.438927 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:11.438967 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:11.441664 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.442390 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.442393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.442396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.442535 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.442940 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.442961 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.442963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.443251 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:11.443376 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:11.444150 (+ 774us) server_negotiation.cc:299] Negotiation successful
0809 22:46:11.444340 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":229,"threads_started":1}
I20260809 22:46:11.445708 32343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:11.426746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:40205 (local address 127.24.136.3:36751)
0809 22:46:11.427637 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:11.427681 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:11.429436 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:11.430244 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:11.430260 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:11.430848 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:11.431483 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:11.431513 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.432928 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.432932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:11.434442 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:11.434462 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.435338 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.435346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:11.435622 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:11.436230 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:11.436255 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:11.438632 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:11.441808 (+ 3176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:11.441815 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:11.441830 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:11.442287 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:11.442630 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:11.442634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:11.442636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:11.442851 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:11.443361 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:11.443368 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:11.443787 (+ 419us) client_negotiation.cc:769] Sending connection context
0809 22:46:11.444031 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 22:46:11.444457 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":309,"threads_started":1}
I20260809 22:46:11.447196 32339 heartbeater.cc:347] Connected to a master server at 127.24.136.62:40205
I20260809 22:46:11.447567 32339 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:11.448356 32339 heartbeater.cc:511] Master 127.24.136.62:40205 requested a full tablet report, sending...
I20260809 22:46:11.449827 31572 ts_manager.cc:194] Registered new tserver with Master: 67032a6f49254488bd119fdf7b648928 (127.24.136.3:40883)
I20260809 22:46:11.450793 31572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:36751
I20260809 22:46:11.460434 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:46:11 dist-test-slave-pww0 krb5kdc[31498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:46:11 dist-test-slave-pww0 krb5kdc[31498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:46:11.521114 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:11.505022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.24.136.1:42631)
0809 22:46:11.505781 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:11.505788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:11.505815 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:46:11.505877 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:11.505884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:11.506107 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:11.506276 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:11.507883 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.508949 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.510286 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.510895 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.511019 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:11.515142 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:11.515158 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:11.515176 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:11.515215 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:11.518130 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.518836 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.518843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.518846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.518984 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.519382 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.519386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.519388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.519814 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:11.520129 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:11.520448 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 22:46:11.520737 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":409,"threads_started":1}
Aug 09 22:46:11 dist-test-slave-pww0 krb5kdc[31498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:11.541531 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:11.526520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.24.136.62:40205)
0809 22:46:11.526771 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:11.526777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:11.526977 (+ 200us) server_negotiation.cc:407] Connection header received
0809 22:46:11.527196 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:11.527201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:11.527279 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:11.527372 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:11.528929 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.529793 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.531201 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.531746 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.531875 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:11.535915 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:11.535922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:11.535925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:11.535968 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:11.538573 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.539304 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.539310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.539314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.539515 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.539890 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.539894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.539897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.540301 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:11.540498 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:11.540938 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 22:46:11.541202 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 22:46:11.542511 31572 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37402
I20260809 22:46:11.543843 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31680
I20260809 22:46:11.559818 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:11.544774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43044 (local address 127.24.136.1:42631)
0809 22:46:11.544949 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:11.544953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:11.547961 (+ 3008us) server_negotiation.cc:407] Connection header received
0809 22:46:11.548134 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:11.548140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:11.548255 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:11.548345 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:11.549636 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.550366 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.551666 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.552095 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.552213 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:11.554554 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:11.554559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:11.554562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:11.554600 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:11.557091 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.557846 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.557852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.557855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.558001 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.558384 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.558390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.558394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.558655 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:11.558880 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:11.559250 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 22:46:11.559509 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:46:11.560463 31893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:11.644905 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31680
I20260809 22:46:11.667506 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31914
Aug 09 22:46:11 dist-test-slave-pww0 krb5kdc[31498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:46:11.683843 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:11.668453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56470 (local address 127.24.136.2:45013)
0809 22:46:11.668981 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:11.668989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:11.669017 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:46:11.669079 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:11.669086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:11.669327 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:11.669472 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:11.670657 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.671764 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.673041 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.673629 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.673733 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:11.677928 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:11.677944 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:11.677960 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:11.678006 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:11.680980 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.681661 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.681669 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.681672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.681814 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.682159 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.682163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.682165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.682564 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:11.682844 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:11.683144 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 22:46:11.683379 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":249,"threads_started":1}
I20260809 22:46:11.685153 32111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:11.768265 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31914
I20260809 22:46:11.790848 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 32128
Aug 09 22:46:11 dist-test-slave-pww0 krb5kdc[31498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:46:11.807576 32367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:11.791783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.24.136.3:40883)
0809 22:46:11.792402 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:11.792409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:11.792439 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:46:11.792502 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:11.792509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:11.792725 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:11.792885 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:11.794244 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.795295 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.796697 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:11.797280 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:11.797412 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:11.801789 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:11.801805 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:11.801822 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:11.801868 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:11.804622 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.805354 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.805363 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.805365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.805508 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:11.805893 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:11.805899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:11.805901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:11.806280 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:11.806601 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:11.806924 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 22:46:11.807204 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":276,"threads_started":1}
I20260809 22:46:11.809010 32331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:11.892158 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 32128
I20260809 22:46:11.913275 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31519
I20260809 22:46:11.914597 31632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:12.020407 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31519
2026-08-09T22:46:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4266 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:12 dist-test-slave-pww0 krb5kdc[32377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:12 dist-test-slave-pww0 krb5kdc[32377](info): set up 2 sockets
Aug 09 22:46:12 dist-test-slave-pww0 krb5kdc[32377](info): commencing operation
krb5kdc: starting...
W20260809 22:46:13.273207 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:46:13 dist-test-slave-pww0 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:46:13.517057 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:34301
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:34301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:13.866675 32396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:13.867030 32396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:13.867180 32396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:13.874914 32396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:13.875058 32396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:13.875139 32396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:13.875226 32396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:13.875293 32396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:13.885861 32396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:34301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:34301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:13.887269 32396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:13.888638 32396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:13.900974 32409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:13.900997 32407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:13.901270 32404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:13.901798 32396 server_base.cc:1034] running on GCE node
I20260809 22:46:13.902694 32396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:13.904718 32396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:13.905987 32396 hybrid_clock.cc:648] HybridClock initialized: now 1786315573905902 us; error 72 us; skew 500 ppm
I20260809 22:46:13.906438 32396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:13 dist-test-slave-pww0 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315573, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:13.910737 32396 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:46:13.913959 32396 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:13.914130 32396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:13.914635 32396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:13.918417 32396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "6a4b0dce2b6643bf91c96ed3bb619475"
format_stamp: "Formatted at 2026-08-09 22:46:13 on dist-test-slave-pww0"
server_key: "eaf77eee41d71b1608742df96f794a3d"
server_key_iv: "1814e1448243b2a32ba427527529dae6"
server_key_version: "encryptionkey@0"
I20260809 22:46:13.919247 32396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "6a4b0dce2b6643bf91c96ed3bb619475"
format_stamp: "Formatted at 2026-08-09 22:46:13 on dist-test-slave-pww0"
server_key: "eaf77eee41d71b1608742df96f794a3d"
server_key_iv: "1814e1448243b2a32ba427527529dae6"
server_key_version: "encryptionkey@0"
I20260809 22:46:13.924974 32396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:46:13.929071 32417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:13.930699 32396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:13.930899 32396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "6a4b0dce2b6643bf91c96ed3bb619475"
format_stamp: "Formatted at 2026-08-09 22:46:13 on dist-test-slave-pww0"
server_key: "eaf77eee41d71b1608742df96f794a3d"
server_key_iv: "1814e1448243b2a32ba427527529dae6"
server_key_version: "encryptionkey@0"
I20260809 22:46:13.931140 32396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:13.945824 32396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:13.950584 32396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:13.950909 32396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:13.969795 32396 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:34301
I20260809 22:46:13.969815 32517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:34301 every 8 connection(s)
I20260809 22:46:13.970808 32396 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:13.971762 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 32396
I20260809 22:46:13.972013 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:13.972680 25120 external_mini_cluster.cc:1428] Setting key c0dd54c46bfd313c225e07d345536017
I20260809 22:46:13.975441 32518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:46:13 dist-test-slave-pww0 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:13.989467 32518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475: Bootstrap starting.
I20260809 22:46:13.993181 32518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:13.994311 32518 log.cc:824] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:13.997695 32518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475: No bootstrap required, opened a new log
I20260809 22:46:13.997710 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:13.974786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.24.136.62:34301)
0809 22:46:13.975680 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:13.975693 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:13.975739 (+ 46us) server_negotiation.cc:407] Connection header received
0809 22:46:13.977463 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:13.977489 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:13.977924 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:13.978316 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:13.980435 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:13.982012 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:13.983865 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:13.984571 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:13.984724 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:13.989718 (+ 4994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:13.989729 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:13.989746 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:13.989800 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:13.993119 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:13.993979 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:13.993987 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:13.993995 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:13.994196 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:13.994589 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:13.994593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:13.994595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:13.995388 (+ 793us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:13.995754 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:13.996214 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 22:46:13.996627 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":391,"threads_started":1}
I20260809 22:46:14.006310 32518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b0dce2b6643bf91c96ed3bb619475" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34301 } }
I20260809 22:46:14.006762 32518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:14.006863 32518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b0dce2b6643bf91c96ed3bb619475, State: Initialized, Role: FOLLOWER
I20260809 22:46:14.007521 32518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b0dce2b6643bf91c96ed3bb619475" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34301 } }
I20260809 22:46:14.007797 32518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:14.007927 32518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:14.008090 32518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:14.010435 32518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b0dce2b6643bf91c96ed3bb619475" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34301 } }
I20260809 22:46:14.011010 32518 leader_election.cc:302] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a4b0dce2b6643bf91c96ed3bb619475; no voters:
I20260809 22:46:14.011715 32518 leader_election.cc:288] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:14.011898 32523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:14.013430 32523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b0dce2b6643bf91c96ed3bb619475, State: Running, Role: LEADER
I20260809 22:46:14.014118 32523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b0dce2b6643bf91c96ed3bb619475" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34301 } }
I20260809 22:46:14.014484 32518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:14.017316 32524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a4b0dce2b6643bf91c96ed3bb619475" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b0dce2b6643bf91c96ed3bb619475" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34301 } } }
I20260809 22:46:14.017289 32526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4b0dce2b6643bf91c96ed3bb619475. Latest consensus state: current_term: 1 leader_uuid: "6a4b0dce2b6643bf91c96ed3bb619475" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b0dce2b6643bf91c96ed3bb619475" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34301 } } }
I20260809 22:46:14.017791 32524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:14.017795 32526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:14.024822 32538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:14.030665 32538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:14.040617 32538 catalog_manager.cc:1347] Generated new cluster ID: a6fa763ccfe24db0b16f25e8bb815988
I20260809 22:46:14.040766 32538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:14.065698 32538 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:14.067145 32538 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:14.076732 32538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475: Generated new TSK 0
I20260809 22:46:14.077689 32538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:46:14.138939 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34301
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:14.494074 32560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:14.494392 32560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:14.494526 32560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:14.502138 32560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:14.502275 32560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:14.502427 32560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:14.512776 32560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:14.514257 32560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:14.515734 32560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:14.530148 32579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:14.530220 32574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:14.530196 32575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:14.536129 32560 server_base.cc:1034] running on GCE node
I20260809 22:46:14.536923 32560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:14.538002 32560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:14.539261 32560 hybrid_clock.cc:648] HybridClock initialized: now 1786315574539178 us; error 71 us; skew 500 ppm
I20260809 22:46:14.539703 32560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:14 dist-test-slave-pww0 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:14.543021 32560 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:46:14.544446 32560 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:14.544569 32560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:14.544955 32560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:14.548288 32560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9"
format_stamp: "Formatted at 2026-08-09 22:46:14 on dist-test-slave-pww0"
server_key: "b466d15a1ca5296702020d94a745d2cc"
server_key_iv: "7b582a1438c0ae8c2bcd1a1088d13373"
server_key_version: "encryptionkey@0"
I20260809 22:46:14.549212 32560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9"
format_stamp: "Formatted at 2026-08-09 22:46:14 on dist-test-slave-pww0"
server_key: "b466d15a1ca5296702020d94a745d2cc"
server_key_iv: "7b582a1438c0ae8c2bcd1a1088d13373"
server_key_version: "encryptionkey@0"
I20260809 22:46:14.556015 32560 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 22:46:14.560073 32591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:14.561688 32560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:46:14.561902 32560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9"
format_stamp: "Formatted at 2026-08-09 22:46:14 on dist-test-slave-pww0"
server_key: "b466d15a1ca5296702020d94a745d2cc"
server_key_iv: "7b582a1438c0ae8c2bcd1a1088d13373"
server_key_version: "encryptionkey@0"
I20260809 22:46:14.562148 32560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:14.585151 32560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:14.589778 32560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:14.590060 32560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:14.591127 32560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:14.592705 32560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:14.592814 32560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:14.592939 32560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:14.593012 32560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:14.625375 32560 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:34011
I20260809 22:46:14.625432 340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:34011 every 8 connection(s)
I20260809 22:46:14.626320 32560 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:14.636505 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 32560
I20260809 22:46:14.636759 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:14.637495 25120 external_mini_cluster.cc:1428] Setting key 9e4cfb70368f034d282827be8d6ff8e6
Aug 09 22:46:14 dist-test-slave-pww0 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:14.652774 344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:14.628879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:55483 (local address 127.24.136.62:34301)
0809 22:46:14.629334 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:14.629338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:14.630397 (+ 1059us) server_negotiation.cc:407] Connection header received
0809 22:46:14.632362 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:14.632367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:14.632459 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:14.632557 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:14.634262 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:14.635067 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:14.637184 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:14.637686 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:14.637825 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:14.646040 (+ 8215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:14.646046 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:14.646050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:14.646094 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:14.648972 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:14.649969 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:14.649975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:14.649977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:14.650132 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:14.650682 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:14.650686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:14.650688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:14.651033 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:14.651197 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:14.652108 (+ 911us) server_negotiation.cc:299] Negotiation successful
0809 22:46:14.652392 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
I20260809 22:46:14.653851 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:14.629460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34301 (local address 127.24.136.1:55483)
0809 22:46:14.630199 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:14.630252 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:14.632038 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:14.632836 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:14.632846 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:14.633444 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:14.634058 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:14.634079 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:14.635236 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:14.635240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:14.636980 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:14.636997 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:14.637969 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:14.637979 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:14.638270 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:14.643020 (+ 4750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:14.643064 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:14.645702 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:14.649215 (+ 3513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:14.649224 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:14.649242 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:14.649790 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:14.650289 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:14.650293 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:14.650296 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:14.650530 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:14.651194 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:14.651204 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:14.651658 (+ 454us) client_negotiation.cc:769] Sending connection context
0809 22:46:14.651940 (+ 282us) client_negotiation.cc:240] Negotiation successful
0809 22:46:14.652564 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":302,"threads_started":1}
I20260809 22:46:14.655718 341 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34301
I20260809 22:46:14.656136 341 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:14.657228 341 heartbeater.cc:511] Master 127.24.136.62:34301 requested a full tablet report, sending...
I20260809 22:46:14.660203 32447 ts_manager.cc:194] Registered new tserver with Master: 0b6775ebef4d4a07a4b6212eaebc1dd9 (127.24.136.1:34011)
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
I20260809 22:46:14.662007 32447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:55483
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:46:14.691912 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34301
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:15.043911 353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:15.044237 353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:15.044338 353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:15.051719 353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:15.051851 353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:15.051970 353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:15.061981 353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:15.063342 353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:15.064733 353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:15.079028 360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:15.079075 367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:15.079113 370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:15.079672 353 server_base.cc:1034] running on GCE node
I20260809 22:46:15.080425 353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:15.081532 353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:15.082787 353 hybrid_clock.cc:648] HybridClock initialized: now 1786315575082719 us; error 54 us; skew 500 ppm
I20260809 22:46:15.083163 353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:15 dist-test-slave-pww0 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:15.086558 353 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:46:15.087898 353 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:15.088011 353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:15.088343 353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:15.091441 353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "d7b3954c469f43b096e0c4789f3997ae"
format_stamp: "Formatted at 2026-08-09 22:46:15 on dist-test-slave-pww0"
server_key: "6a584bef8233938924eb4d2d7cf8ec40"
server_key_iv: "f305845e354c93d46a7d87f92944e2db"
server_key_version: "encryptionkey@0"
I20260809 22:46:15.092404 353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "d7b3954c469f43b096e0c4789f3997ae"
format_stamp: "Formatted at 2026-08-09 22:46:15 on dist-test-slave-pww0"
server_key: "6a584bef8233938924eb4d2d7cf8ec40"
server_key_iv: "f305845e354c93d46a7d87f92944e2db"
server_key_version: "encryptionkey@0"
I20260809 22:46:15.098039 353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:46:15.102089 380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:15.103657 353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:15.103869 353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "d7b3954c469f43b096e0c4789f3997ae"
format_stamp: "Formatted at 2026-08-09 22:46:15 on dist-test-slave-pww0"
server_key: "6a584bef8233938924eb4d2d7cf8ec40"
server_key_iv: "f305845e354c93d46a7d87f92944e2db"
server_key_version: "encryptionkey@0"
I20260809 22:46:15.104127 353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:15.124890 353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:15.130306 353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:15.130681 353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:15.134613 353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:15.136628 353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:15.136802 353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:46:15.136953 353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:15.137038 353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:15.170411 353 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:44571
I20260809 22:46:15.170461 580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:44571 every 8 connection(s)
I20260809 22:46:15.171322 353 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:15.178048 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 353
I20260809 22:46:15.178249 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:15.178663 25120 external_mini_cluster.cc:1428] Setting key 407261c5a819b9a30ec1670756d2c66a
Aug 09 22:46:15 dist-test-slave-pww0 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:15.192694 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.173956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:55501 (local address 127.24.136.62:34301)
0809 22:46:15.174410 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:15.174415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:15.175545 (+ 1130us) server_negotiation.cc:407] Connection header received
0809 22:46:15.177455 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:15.177461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:15.177544 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:15.177640 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:15.179520 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.180299 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.181923 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.182358 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.182475 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.186194 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:15.186200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:15.186204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:15.186240 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:15.189125 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:15.189966 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:15.189971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:15.189974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:15.190143 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:15.190590 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:15.190595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:15.190598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:15.190994 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:15.191167 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:15.192056 (+ 889us) server_negotiation.cc:299] Negotiation successful
0809 22:46:15.192313 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":223,"threads_started":1}
I20260809 22:46:15.193593 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.174556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34301 (local address 127.24.136.2:55501)
0809 22:46:15.175342 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:15.175389 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:15.177132 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:15.177902 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:15.177911 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:15.178547 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:15.179241 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.179271 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.180460 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.180464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.181763 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.181777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.182595 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.182603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.182835 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.183576 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:15.183613 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:15.185931 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:15.189306 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:15.189314 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:15.189331 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:15.189825 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:15.190263 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:15.190267 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:15.190269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:15.190481 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:15.191192 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:15.191203 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:15.191650 (+ 447us) client_negotiation.cc:769] Sending connection context
0809 22:46:15.191923 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 22:46:15.192402 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":322,"threads_started":1}
I20260809 22:46:15.195289 583 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34301
I20260809 22:46:15.195720 583 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:15.196570 583 heartbeater.cc:511] Master 127.24.136.62:34301 requested a full tablet report, sending...
I20260809 22:46:15.198559 32447 ts_manager.cc:194] Registered new tserver with Master: d7b3954c469f43b096e0c4789f3997ae (127.24.136.2:44571)
I20260809 22:46:15.199451 32447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:55501
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:46:15.235237 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34301
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:15.585481 600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:15.585824 600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:15.585970 600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:15.593554 600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:15.593714 600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:15.593894 600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:15.604450 600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:15.605934 600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:15.607369 600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:15.621526 613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:15.621603 608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:15.621497 606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:15.622466 600 server_base.cc:1034] running on GCE node
I20260809 22:46:15.623183 600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:15.624368 600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:15.625654 600 hybrid_clock.cc:648] HybridClock initialized: now 1786315575625517 us; error 94 us; skew 500 ppm
I20260809 22:46:15.626077 600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:15 dist-test-slave-pww0 krb5kdc[32377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:15.630306 600 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:46:15.631851 600 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:15.631961 600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:15.632249 600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:15.635707 600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "1c0744e562a54bbdad29837f513fd7ab"
format_stamp: "Formatted at 2026-08-09 22:46:15 on dist-test-slave-pww0"
server_key: "f4c798928ef8a6fff787908f5186e9da"
server_key_iv: "6af63985dc84353b2d540a35bf325014"
server_key_version: "encryptionkey@0"
I20260809 22:46:15.636656 600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "1c0744e562a54bbdad29837f513fd7ab"
format_stamp: "Formatted at 2026-08-09 22:46:15 on dist-test-slave-pww0"
server_key: "f4c798928ef8a6fff787908f5186e9da"
server_key_iv: "6af63985dc84353b2d540a35bf325014"
server_key_version: "encryptionkey@0"
I20260809 22:46:15.642755 600 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 22:46:15.646849 623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:15.648559 600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:46:15.648751 600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "1c0744e562a54bbdad29837f513fd7ab"
format_stamp: "Formatted at 2026-08-09 22:46:15 on dist-test-slave-pww0"
server_key: "f4c798928ef8a6fff787908f5186e9da"
server_key_iv: "6af63985dc84353b2d540a35bf325014"
server_key_version: "encryptionkey@0"
I20260809 22:46:15.649016 600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:15.665175 600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:15.665702 341 heartbeater.cc:503] Master 127.24.136.62:34301 was elected leader, sending a full tablet report...
I20260809 22:46:15.670135 600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:15.670433 600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:15.671571 600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:15.673046 600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:15.673137 600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:15.673254 600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:15.673343 600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:15.705318 600 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:45475
I20260809 22:46:15.705366 839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:45475 every 8 connection(s)
I20260809 22:46:15.706216 600 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:15.710341 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 600
I20260809 22:46:15.710552 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:15.711143 25120 external_mini_cluster.cc:1428] Setting key deedb2b8a4d28cd5ddadbaa57bacc3f0
Aug 09 22:46:15 dist-test-slave-pww0 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:15.729009 844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.708989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:45575 (local address 127.24.136.62:34301)
0809 22:46:15.709534 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:15.709541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:15.711316 (+ 1775us) server_negotiation.cc:407] Connection header received
0809 22:46:15.713712 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:15.713716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:15.713821 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:15.713908 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:15.715870 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.716671 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.718338 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.718850 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.718957 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.722614 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:15.722620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:15.722623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:15.722662 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:15.725563 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:15.726592 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:15.726598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:15.726601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:15.726769 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:15.727310 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:15.727313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:15.727315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:15.727661 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:15.727788 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:15.728536 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 22:46:15.728738 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":283,"threads_started":1}
I20260809 22:46:15.730069 845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.709636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34301 (local address 127.24.136.3:45575)
0809 22:46:15.711098 (+ 1462us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:15.711167 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:15.713399 (+ 2232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:15.714233 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:15.714247 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:15.714942 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:15.715647 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.715669 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.716826 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.716830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.718154 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.718168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.719081 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.719087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.719310 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.720022 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:15.720051 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:15.722367 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:15.725732 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:15.725745 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:15.725763 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:15.726437 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:15.726902 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:15.726906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:15.726909 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:15.727196 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:15.727801 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:15.727808 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:15.728175 (+ 367us) client_negotiation.cc:769] Sending connection context
0809 22:46:15.728425 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 22:46:15.728877 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1142,"thread_start_us":912,"threads_started":1}
I20260809 22:46:15.731613 841 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34301
I20260809 22:46:15.731912 841 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:15.732601 841 heartbeater.cc:511] Master 127.24.136.62:34301 requested a full tablet report, sending...
I20260809 22:46:15.734107 32447 ts_manager.cc:194] Registered new tserver with Master: 1c0744e562a54bbdad29837f513fd7ab (127.24.136.3:45475)
I20260809 22:46:15.734982 32447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:45575
I20260809 22:46:15.739817 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:46:15.758092 844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.744733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.24.136.62:34301)
0809 22:46:15.744975 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:15.744982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:15.745308 (+ 326us) server_negotiation.cc:407] Connection header received
0809 22:46:15.745494 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:15.745498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:15.745567 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:15.745650 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:15.747132 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.747963 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.749368 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.749801 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.749925 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.752512 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:15.752518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:15.752522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:15.752565 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:15.755227 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:15.756013 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:15.756017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:15.756020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:15.756167 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:15.756550 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:15.756554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:15.756556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:15.756910 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:15.757164 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:15.757505 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 22:46:15.757763 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:46:15.762696 32447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:46:15.769925 32447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:15.805675 867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.784036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.24.136.1:34011)
0809 22:46:15.796570 (+ 12534us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:15.796582 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:15.797378 (+ 796us) server_negotiation.cc:407] Connection header received
0809 22:46:15.797515 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:15.797524 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:15.797821 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:15.798874 (+ 1053us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:15.799097 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.800565 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.803105 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.804465 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.804600 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.804870 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:15.805156 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 22:46:15.805331 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12334,"thread_start_us":13069,"threads_started":1}
I20260809 22:46:15.805734 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.783756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:34011 (local address 127.0.0.1:43642)
0809 22:46:15.796571 (+ 12815us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:15.796625 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:15.796904 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:15.798216 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:15.798223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:15.798255 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:15.798686 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.798700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.800742 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.800747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.802911 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.802930 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.804686 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.804692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.804900 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.804959 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 22:46:15.805080 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 22:46:15.805307 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12458,"mutex_wait_us":65,"thread_start_us":13736,"threads_started":1}
I20260809 22:46:15.806921 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.784392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.24.136.3:45475)
0809 22:46:15.797145 (+ 12753us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:15.797154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:15.798108 (+ 954us) server_negotiation.cc:407] Connection header received
0809 22:46:15.798222 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:15.798230 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:15.798461 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:15.798672 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:15.799707 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.801313 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.803977 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.805426 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.805555 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.805790 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:15.806389 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 22:46:15.806573 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12530,"thread_start_us":12750,"threads_started":1}
I20260809 22:46:15.807803 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.783944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:45475 (local address 127.0.0.1:47804)
0809 22:46:15.797826 (+ 13882us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:15.797846 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:15.797981 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:15.798932 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:15.798937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:15.798963 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:15.799482 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.799497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.801494 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.801499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.803784 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.803800 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.805938 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.805944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.806129 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.806168 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:46:15.807220 (+ 1052us) client_negotiation.cc:240] Negotiation successful
0809 22:46:15.807387 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13738,"mutex_wait_us":24,"thread_start_us":12066,"threads_started":1}
I20260809 22:46:15.809080 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.798898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:44571 (local address 127.0.0.1:43050)
0809 22:46:15.799573 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:15.799591 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:15.799715 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:15.800524 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:15.800529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:15.800554 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:15.801036 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.801049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.802980 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.802984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.806283 (+ 3299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.806298 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.808182 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.808189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.808363 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.808396 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:46:15.808492 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:46:15.808627 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":323,"threads_started":1}
I20260809 22:46:15.809546 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.798248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.24.136.2:44571)
0809 22:46:15.798837 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:15.798845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:15.799707 (+ 862us) server_negotiation.cc:407] Connection header received
0809 22:46:15.799862 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:15.799870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:15.800092 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:15.800290 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:15.801259 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.802821 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.806480 (+ 3659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.807978 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.808125 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.808338 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:15.808568 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 22:46:15.809182 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":331,"threads_started":1}
I20260809 22:46:15.810437 32689 tablet_service.cc:1467] Processing CreateTablet for tablet 03b030a2c85a4061afbef455783622d8 (DEFAULT_TABLE table=test-table [id=6aedcd5e969d48f0ab10f4631e7a33cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:15.812235 32689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03b030a2c85a4061afbef455783622d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:15.811915 717 tablet_service.cc:1467] Processing CreateTablet for tablet 03b030a2c85a4061afbef455783622d8 (DEFAULT_TABLE table=test-table [id=6aedcd5e969d48f0ab10f4631e7a33cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:15.813100 717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03b030a2c85a4061afbef455783622d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:15.813894 458 tablet_service.cc:1467] Processing CreateTablet for tablet 03b030a2c85a4061afbef455783622d8 (DEFAULT_TABLE table=test-table [id=6aedcd5e969d48f0ab10f4631e7a33cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:15.816313 458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03b030a2c85a4061afbef455783622d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:15.825559 875 tablet_bootstrap.cc:492] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: Bootstrap starting.
I20260809 22:46:15.827884 876 tablet_bootstrap.cc:492] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: Bootstrap starting.
I20260809 22:46:15.829754 875 tablet_bootstrap.cc:654] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:15.831631 881 tablet_bootstrap.cc:492] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: Bootstrap starting.
I20260809 22:46:15.831790 876 tablet_bootstrap.cc:654] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:15.831955 875 log.cc:824] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:15.833168 876 log.cc:824] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:15.835553 875 tablet_bootstrap.cc:492] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: No bootstrap required, opened a new log
I20260809 22:46:15.835638 881 tablet_bootstrap.cc:654] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:15.835911 875 ts_tablet_manager.cc:1397] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260809 22:46:15.836869 876 tablet_bootstrap.cc:492] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: No bootstrap required, opened a new log
I20260809 22:46:15.836838 881 log.cc:824] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:15.837177 876 ts_tablet_manager.cc:1397] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 22:46:15.846230 881 tablet_bootstrap.cc:492] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: No bootstrap required, opened a new log
I20260809 22:46:15.846560 881 ts_tablet_manager.cc:1397] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.003s
I20260809 22:46:15.847647 876 raft_consensus.cc:348] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } }
I20260809 22:46:15.847625 875 raft_consensus.cc:348] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } }
I20260809 22:46:15.848152 875 raft_consensus.cc:374] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:15.848152 876 raft_consensus.cc:374] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:15.848284 876 raft_consensus.cc:729] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6775ebef4d4a07a4b6212eaebc1dd9, State: Initialized, Role: FOLLOWER
I20260809 22:46:15.848320 875 raft_consensus.cc:729] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c0744e562a54bbdad29837f513fd7ab, State: Initialized, Role: FOLLOWER
I20260809 22:46:15.849014 876 consensus_queue.cc:260] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } }
I20260809 22:46:15.849014 875 consensus_queue.cc:260] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } }
I20260809 22:46:15.853924 841 heartbeater.cc:503] Master 127.24.136.62:34301 was elected leader, sending a full tablet report...
I20260809 22:46:15.855379 876 ts_tablet_manager.cc:1428] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260809 22:46:15.855919 881 raft_consensus.cc:348] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } }
I20260809 22:46:15.856556 881 raft_consensus.cc:374] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:15.856667 875 ts_tablet_manager.cc:1428] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: Time spent starting tablet: real 0.021s user 0.013s sys 0.004s
I20260809 22:46:15.856721 881 raft_consensus.cc:729] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b3954c469f43b096e0c4789f3997ae, State: Initialized, Role: FOLLOWER
I20260809 22:46:15.857491 881 consensus_queue.cc:260] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } }
I20260809 22:46:15.858728 583 heartbeater.cc:503] Master 127.24.136.62:34301 was elected leader, sending a full tablet report...
I20260809 22:46:15.859403 881 ts_tablet_manager.cc:1428] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: Time spent starting tablet: real 0.013s user 0.005s sys 0.008s
W20260809 22:46:15.878717 343 tablet.cc:2369] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:46:15.923374 585 tablet.cc:2369] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:46:15.957819 843 tablet.cc:2369] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:15.958106 887 raft_consensus.cc:482] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:15.958451 887 raft_consensus.cc:504] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } }
I20260809 22:46:15.960281 887 leader_election.cc:288] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b6775ebef4d4a07a4b6212eaebc1dd9 (127.24.136.1:34011), d7b3954c469f43b096e0c4789f3997ae (127.24.136.2:44571)
I20260809 22:46:15.966241 867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.960833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:44471 (local address 127.24.136.1:34011)
0809 22:46:15.961040 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:15.961045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:15.961065 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:15.961139 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:15.961144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:15.961220 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:15.961315 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:15.961885 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.962953 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.964577 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.965343 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.965497 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.965577 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:15.965852 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 22:46:15.965982 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:46:15.966300 845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.960642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:34011 (local address 127.24.136.3:44471)
0809 22:46:15.960845 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:15.960863 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:15.961001 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:15.961376 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:15.961382 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:15.961425 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:15.961697 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.961709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.963081 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.963085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.964415 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.964428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.965502 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.965508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.965684 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.965719 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:46:15.965815 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:46:15.965953 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:46:15.967279 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.961304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:38391 (local address 127.24.136.2:44571)
0809 22:46:15.961508 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:15.961515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:15.962010 (+ 495us) server_negotiation.cc:407] Connection header received
0809 22:46:15.962158 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:15.962162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:15.962227 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:15.962299 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:15.962905 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.963702 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.965398 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.966281 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.966430 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.966499 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:15.966954 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 22:46:15.967062 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:46:15.967389 891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.961105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:44571 (local address 127.24.136.3:38391)
0809 22:46:15.961897 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:15.961915 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:15.962044 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:15.962340 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:15.962345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:15.962374 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:15.962690 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.962702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.963860 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.963866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.965234 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:15.965248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:15.966639 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:15.966645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:15.966776 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:15.966815 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:46:15.966915 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 22:46:15.967054 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":502,"threads_started":1}
I20260809 22:46:15.967484 32728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b030a2c85a4061afbef455783622d8" candidate_uuid: "1c0744e562a54bbdad29837f513fd7ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" is_pre_election: true
I20260809 22:46:15.968116 32728 raft_consensus.cc:2454] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c0744e562a54bbdad29837f513fd7ab in term 0.
I20260809 22:46:15.968444 496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b030a2c85a4061afbef455783622d8" candidate_uuid: "1c0744e562a54bbdad29837f513fd7ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b3954c469f43b096e0c4789f3997ae" is_pre_election: true
I20260809 22:46:15.968891 637 leader_election.cc:302] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6775ebef4d4a07a4b6212eaebc1dd9, 1c0744e562a54bbdad29837f513fd7ab; no voters:
I20260809 22:46:15.968991 496 raft_consensus.cc:2454] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c0744e562a54bbdad29837f513fd7ab in term 0.
I20260809 22:46:15.971628 887 raft_consensus.cc:2781] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:15.971800 887 raft_consensus.cc:482] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:15.971913 887 raft_consensus.cc:3037] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:15.974186 887 raft_consensus.cc:504] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } }
I20260809 22:46:15.974856 887 leader_election.cc:288] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [CANDIDATE]: Term 1 election: Requested vote from peers 0b6775ebef4d4a07a4b6212eaebc1dd9 (127.24.136.1:34011), d7b3954c469f43b096e0c4789f3997ae (127.24.136.2:44571)
I20260809 22:46:15.975574 496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b030a2c85a4061afbef455783622d8" candidate_uuid: "1c0744e562a54bbdad29837f513fd7ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b3954c469f43b096e0c4789f3997ae"
I20260809 22:46:15.975572 32728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b030a2c85a4061afbef455783622d8" candidate_uuid: "1c0744e562a54bbdad29837f513fd7ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9"
I20260809 22:46:15.975816 496 raft_consensus.cc:3037] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:15.975832 32728 raft_consensus.cc:3037] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:15.978186 496 raft_consensus.cc:2454] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c0744e562a54bbdad29837f513fd7ab in term 1.
I20260809 22:46:15.978722 637 leader_election.cc:302] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c0744e562a54bbdad29837f513fd7ab, d7b3954c469f43b096e0c4789f3997ae; no voters:
I20260809 22:46:15.979044 887 raft_consensus.cc:2781] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:15.979175 32728 raft_consensus.cc:2454] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c0744e562a54bbdad29837f513fd7ab in term 1.
I20260809 22:46:15.979713 887 raft_consensus.cc:686] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 1 LEADER]: Becoming Leader. State: Replica: 1c0744e562a54bbdad29837f513fd7ab, State: Running, Role: LEADER
I20260809 22:46:15.980374 887 consensus_queue.cc:237] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } }
I20260809 22:46:15.985760 32440 catalog_manager.cc:5495] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c0744e562a54bbdad29837f513fd7ab (127.24.136.3). New cstate: current_term: 1 leader_uuid: "1c0744e562a54bbdad29837f513fd7ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c0744e562a54bbdad29837f513fd7ab" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 } health_report { overall_health: UNKNOWN } } }
I20260809 22:46:16.004701 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:15.999393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.24.136.3:45475)
0809 22:46:15.999605 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:15.999610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:15.999629 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:46:15.999764 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:15.999768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:15.999850 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:15.999986 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:16.000351 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:16.001081 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:16.002463 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:16.002883 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:16.003001 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:16.003541 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:16.004122 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:16.004245 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:16.004390 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 22:46:16.004472 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:46:16.013926 496 raft_consensus.cc:1261] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [term 1 FOLLOWER]: Refusing update from remote peer 1c0744e562a54bbdad29837f513fd7ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:16.013926 32728 raft_consensus.cc:1261] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [term 1 FOLLOWER]: Refusing update from remote peer 1c0744e562a54bbdad29837f513fd7ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:16.015091 887 consensus_queue.cc:1035] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6775ebef4d4a07a4b6212eaebc1dd9" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 34011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:16.015440 893 consensus_queue.cc:1035] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7b3954c469f43b096e0c4789f3997ae" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:16.043237 32440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:16.043555 32440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:16.047186 32440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a4b0dce2b6643bf91c96ed3bb619475: Sending DeleteTablet for 3 replicas of tablet 03b030a2c85a4061afbef455783622d8
I20260809 22:46:16.048599 717 tablet_service.cc:1514] Processing DeleteTablet for tablet 03b030a2c85a4061afbef455783622d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:16 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:47804
I20260809 22:46:16.049044 458 tablet_service.cc:1514] Processing DeleteTablet for tablet 03b030a2c85a4061afbef455783622d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:16 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:43050
I20260809 22:46:16.049407 32689 tablet_service.cc:1514] Processing DeleteTablet for tablet 03b030a2c85a4061afbef455783622d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:16 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:43642
I20260809 22:46:16.049973 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 32560
I20260809 22:46:16.064389 915 tablet_replica.cc:330] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: stopping tablet replica
I20260809 22:46:16.065142 915 raft_consensus.cc:2229] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:16.065266 914 tablet_replica.cc:330] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: stopping tablet replica
I20260809 22:46:16.065907 915 raft_consensus.cc:2258] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:16.069258 915 ts_tablet_manager.cc:1905] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:16.069926 914 raft_consensus.cc:2229] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:16.070876 914 raft_consensus.cc:2258] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:16.071621 919 tablet_replica.cc:330] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: stopping tablet replica
I20260809 22:46:16.072188 919 raft_consensus.cc:2229] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:16.073769 919 raft_consensus.cc:2258] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:16.075003 914 ts_tablet_manager.cc:1905] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 22:46:16 dist-test-slave-pww0 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:46:16.077895 915 ts_tablet_manager.cc:1918] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:16.078207 915 log.cc:1196] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/03b030a2c85a4061afbef455783622d8
I20260809 22:46:16.078891 915 ts_tablet_manager.cc:1939] T 03b030a2c85a4061afbef455783622d8 P d7b3954c469f43b096e0c4789f3997ae: Deleting consensus metadata
I20260809 22:46:16.079677 919 ts_tablet_manager.cc:1905] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:16.081137 32422 catalog_manager.cc:4841] TS d7b3954c469f43b096e0c4789f3997ae (127.24.136.2:44571): tablet 03b030a2c85a4061afbef455783622d8 (table test-table [id=6aedcd5e969d48f0ab10f4631e7a33cb]) successfully deleted
I20260809 22:46:16.082984 914 ts_tablet_manager.cc:1918] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:16.083225 914 log.cc:1196] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/03b030a2c85a4061afbef455783622d8
I20260809 22:46:16.083878 914 ts_tablet_manager.cc:1939] T 03b030a2c85a4061afbef455783622d8 P 1c0744e562a54bbdad29837f513fd7ab: Deleting consensus metadata
I20260809 22:46:16.085873 32418 catalog_manager.cc:4841] TS 1c0744e562a54bbdad29837f513fd7ab (127.24.136.3:45475): tablet 03b030a2c85a4061afbef455783622d8 (table test-table [id=6aedcd5e969d48f0ab10f4631e7a33cb]) successfully deleted
I20260809 22:46:16.088325 919 ts_tablet_manager.cc:1918] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:16.088537 919 log.cc:1196] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/03b030a2c85a4061afbef455783622d8
I20260809 22:46:16.089100 919 ts_tablet_manager.cc:1939] T 03b030a2c85a4061afbef455783622d8 P 0b6775ebef4d4a07a4b6212eaebc1dd9: Deleting consensus metadata
I20260809 22:46:16.090977 32422 catalog_manager.cc:4841] TS 0b6775ebef4d4a07a4b6212eaebc1dd9 (127.24.136.1:34011): tablet 03b030a2c85a4061afbef455783622d8 (table test-table [id=6aedcd5e969d48f0ab10f4631e7a33cb]) successfully deleted
I20260809 22:46:16.093304 867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:16.052848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.24.136.1:34011)
0809 22:46:16.053062 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:16.053069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:16.062024 (+ 8955us) server_negotiation.cc:407] Connection header received
0809 22:46:16.062557 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:16.062565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:16.062676 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:16.062768 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:16.065564 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:16.066928 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:16.071523 (+ 4595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:16.072265 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:16.072396 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:16.085899 (+ 13503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:16.085917 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:16.085941 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:16.085989 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:16.090127 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:16.091262 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:16.091271 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:16.091273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:16.091424 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:16.091886 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:16.091889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:16.091891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:16.092266 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:16.092513 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:16.092873 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 22:46:16.093043 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:46:16.094019 335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:16.183847 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 32560
I20260809 22:46:16.212716 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 353
Aug 09 22:46:16 dist-test-slave-pww0 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:46:16.230692 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:16.213611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43056 (local address 127.24.136.2:44571)
0809 22:46:16.213797 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:16.213802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:16.213822 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:16.214013 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:16.214017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:16.214085 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:16.214168 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:16.215940 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:16.216857 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:16.218824 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:16.219254 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:16.219409 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:16.224578 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:16.224596 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:16.224612 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:16.224681 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:16.227712 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:16.228456 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:16.228465 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:16.228467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:16.228614 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:16.229047 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:16.229051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:16.229053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:16.229483 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:16.229790 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:16.230103 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 22:46:16.230341 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:46:16.231532 571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:16.321250 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 353
I20260809 22:46:16.345484 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 600
Aug 09 22:46:16 dist-test-slave-pww0 krb5kdc[32377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:46:16.365087 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:16.346518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.24.136.3:45475)
0809 22:46:16.346733 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:16.346740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:16.346762 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:46:16.346848 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:16.346854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:16.346949 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:16.347083 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:16.348413 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:16.349699 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:16.351109 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:16.352004 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:16.352166 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:16.356853 (+ 4687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:16.356877 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:16.356900 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:16.356971 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:16.361353 (+ 4382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:16.362212 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:16.362223 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:16.362227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:16.362455 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:16.362985 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:16.362991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:16.362994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:16.363630 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:16.363993 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:16.364318 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 22:46:16.364619 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:46:16.366050 830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:16.488341 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 600
I20260809 22:46:16.518266 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 32396
I20260809 22:46:16.519510 32512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:16.631021 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 32396
2026-08-09T22:46:16Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4616 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T22:46:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:16Z Disabled control of system clock
I20260809 22:46:16.718103 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:34583
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:34583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 22:46:17.082406 931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:17.082755 931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:17.082860 931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:17.090396 931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:17.090538 931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:17.090577 931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:17.090626 931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:17.090670 931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:17.100854 931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:34583
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:34583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:17.102336 931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:17.104316 931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:17.117110 944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:17.117249 940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:17.117345 941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:17.118036 931 server_base.cc:1034] running on GCE node
I20260809 22:46:17.118947 931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:17.120597 931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:17.121862 931 hybrid_clock.cc:648] HybridClock initialized: now 1786315577121810 us; error 43 us; skew 500 ppm
I20260809 22:46:17.122287 931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:46:17.125011 931 webserver.cc:460] Webserver started at http://127.24.136.62:36261/ using document root <none> and password file <none>
I20260809 22:46:17.126001 931 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:17.126142 931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:17.126566 931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:17.130703 931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "0ec65745fad1410dbebce6e8b29d0c62"
format_stamp: "Formatted at 2026-08-09 22:46:17 on dist-test-slave-pww0"
server_key: "96b0babd3498b821f93f2e05481bce13"
server_key_iv: "a799d2813e3bc993de1c877398bb3116"
server_key_version: "encryptionkey@0"
I20260809 22:46:17.131706 931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "0ec65745fad1410dbebce6e8b29d0c62"
format_stamp: "Formatted at 2026-08-09 22:46:17 on dist-test-slave-pww0"
server_key: "96b0babd3498b821f93f2e05481bce13"
server_key_iv: "a799d2813e3bc993de1c877398bb3116"
server_key_version: "encryptionkey@0"
I20260809 22:46:17.139551 931 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260809 22:46:17.143537 951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:17.145068 931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:17.145279 931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "0ec65745fad1410dbebce6e8b29d0c62"
format_stamp: "Formatted at 2026-08-09 22:46:17 on dist-test-slave-pww0"
server_key: "96b0babd3498b821f93f2e05481bce13"
server_key_iv: "a799d2813e3bc993de1c877398bb3116"
server_key_version: "encryptionkey@0"
I20260809 22:46:17.145514 931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:17.151811 931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:17.152737 931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:17.153138 931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:17.171840 931 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:34583
I20260809 22:46:17.171860 1051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:34583 every 8 connection(s)
I20260809 22:46:17.173386 931 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:17.177651 1055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:17.183184 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 931
I20260809 22:46:17.183390 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:17.184103 25120 external_mini_cluster.cc:1428] Setting key bc9a90971eb2920bd315042f6231e439
I20260809 22:46:17.189284 1055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62: Bootstrap starting.
I20260809 22:46:17.193044 1055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:17.193288 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:17.186386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50622 (local address 127.24.136.62:34583)
0809 22:46:17.187554 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:17.187572 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:17.187632 (+ 60us) server_negotiation.cc:407] Connection header received
0809 22:46:17.189462 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:17.189492 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:17.189963 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:17.190331 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:17.191562 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:17.191568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:17.191584 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:17.191600 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:17.191667 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:17.191689 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:17.191892 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:17.191996 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 22:46:17.192265 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":378,"threads_started":1}
I20260809 22:46:17.194260 1055 log.cc:824] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:17.197644 1055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62: No bootstrap required, opened a new log
I20260809 22:46:17.205667 1055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec65745fad1410dbebce6e8b29d0c62" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34583 } }
I20260809 22:46:17.206162 1055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:17.206271 1055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec65745fad1410dbebce6e8b29d0c62, State: Initialized, Role: FOLLOWER
I20260809 22:46:17.206867 1055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec65745fad1410dbebce6e8b29d0c62" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34583 } }
I20260809 22:46:17.207121 1055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:17.207252 1055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:17.207422 1055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:17.209821 1055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec65745fad1410dbebce6e8b29d0c62" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34583 } }
I20260809 22:46:17.210440 1055 leader_election.cc:302] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ec65745fad1410dbebce6e8b29d0c62; no voters:
I20260809 22:46:17.211045 1055 leader_election.cc:288] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:17.211198 1063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:17.212457 1063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [term 1 LEADER]: Becoming Leader. State: Replica: 0ec65745fad1410dbebce6e8b29d0c62, State: Running, Role: LEADER
I20260809 22:46:17.213029 1063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec65745fad1410dbebce6e8b29d0c62" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34583 } }
I20260809 22:46:17.213435 1055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:17.216334 1068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ec65745fad1410dbebce6e8b29d0c62. Latest consensus state: current_term: 1 leader_uuid: "0ec65745fad1410dbebce6e8b29d0c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec65745fad1410dbebce6e8b29d0c62" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34583 } } }
I20260809 22:46:17.216650 1066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ec65745fad1410dbebce6e8b29d0c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec65745fad1410dbebce6e8b29d0c62" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34583 } } }
I20260809 22:46:17.216923 1068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:17.216997 1066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:17.218024 1083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:17.223975 1083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:17.232760 1083 catalog_manager.cc:1347] Generated new cluster ID: ced49fab5d224bb584f9198547bea661
I20260809 22:46:17.232898 1083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:17.255406 1083 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:17.256016 1083 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:17.268302 1083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62: Generated new TSK 0
I20260809 22:46:17.269543 1083 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:46:17.282430 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34583
--builtin_ntp_servers=127.24.136.20:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 22:46:17.639644 1100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:17.639997 1100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:17.640142 1100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:17.647907 1100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:17.648049 1100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:17.648206 1100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:17.657997 1100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:17.659497 1100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:17.661406 1100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:17.675364 1120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:17.675388 1115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:17.675437 1117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:17.676359 1100 server_base.cc:1034] running on GCE node
I20260809 22:46:17.677016 1100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:17.678099 1100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:17.679355 1100 hybrid_clock.cc:648] HybridClock initialized: now 1786315577679286 us; error 48 us; skew 500 ppm
I20260809 22:46:17.679749 1100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:46:17.682209 1100 webserver.cc:460] Webserver started at http://127.24.136.1:34783/ using document root <none> and password file <none>
I20260809 22:46:17.683015 1100 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:17.683116 1100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:17.683431 1100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:17.686602 1100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "1f87c6b3ed0f43d8be3aa81873c25c55"
format_stamp: "Formatted at 2026-08-09 22:46:17 on dist-test-slave-pww0"
server_key: "df24c27b09bbf40b9d01a17aa0f8a8bb"
server_key_iv: "a1401a64b287ae975e38fd162f281db5"
server_key_version: "encryptionkey@0"
I20260809 22:46:17.687433 1100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "1f87c6b3ed0f43d8be3aa81873c25c55"
format_stamp: "Formatted at 2026-08-09 22:46:17 on dist-test-slave-pww0"
server_key: "df24c27b09bbf40b9d01a17aa0f8a8bb"
server_key_iv: "a1401a64b287ae975e38fd162f281db5"
server_key_version: "encryptionkey@0"
I20260809 22:46:17.693372 1100 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 22:46:17.697408 1131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:17.698949 1100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:46:17.699137 1100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "1f87c6b3ed0f43d8be3aa81873c25c55"
format_stamp: "Formatted at 2026-08-09 22:46:17 on dist-test-slave-pww0"
server_key: "df24c27b09bbf40b9d01a17aa0f8a8bb"
server_key_iv: "a1401a64b287ae975e38fd162f281db5"
server_key_version: "encryptionkey@0"
I20260809 22:46:17.699383 1100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:17.705452 1100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:17.706367 1100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:17.706642 1100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:17.707691 1100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:17.709260 1100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:17.709354 1100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:17.709466 1100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:17.709544 1100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:17.740602 1100 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:33207
I20260809 22:46:17.740653 1332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:33207 every 8 connection(s)
I20260809 22:46:17.742004 1100 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:17.748809 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 1100
I20260809 22:46:17.749027 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:17.749774 25120 external_mini_cluster.cc:1428] Setting key f50ee8512391de21b72b8b508ad28291
I20260809 22:46:17.750844 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:17.744692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:41873 (local address 127.24.136.62:34583)
0809 22:46:17.745173 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:17.745179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:17.746444 (+ 1265us) server_negotiation.cc:407] Connection header received
0809 22:46:17.748242 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:17.748247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:17.748309 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:17.748410 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:17.750031 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:17.750038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:17.750043 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:17.750066 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:17.750123 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:17.750142 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:17.750305 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:17.750457 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 22:46:17.750520 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":226,"threads_started":1}
I20260809 22:46:17.752573 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:17.745313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34583 (local address 127.24.136.1:41873)
0809 22:46:17.746272 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:17.746312 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:17.747959 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:17.748729 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:17.748741 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:17.748883 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:17.749657 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:17.749661 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:17.749719 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:17.749828 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:17.750244 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:17.750255 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:17.750263 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:46:17.750524 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 22:46:17.750835 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":261,"threads_started":1}
I20260809 22:46:17.752750 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34583
--builtin_ntp_servers=127.24.136.20:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 22:46:17.754211 1334 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34583
I20260809 22:46:17.754595 1334 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:17.755373 1334 heartbeater.cc:511] Master 127.24.136.62:34583 requested a full tablet report, sending...
I20260809 22:46:17.757776 983 ts_manager.cc:194] Registered new tserver with Master: 1f87c6b3ed0f43d8be3aa81873c25c55 (127.24.136.1:33207)
W20260809 22:46:18.094506 1340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:18.094821 1340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:18.094938 1340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:18.102312 1340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:18.102440 1340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:18.102589 1340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:18.112084 1340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:18.113555 1340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:18.115705 1340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:18.129503 1347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:18.129432 1348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:18.129479 1352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:18.130291 1340 server_base.cc:1034] running on GCE node
I20260809 22:46:18.130970 1340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:18.132078 1340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:18.133299 1340 hybrid_clock.cc:648] HybridClock initialized: now 1786315578133238 us; error 56 us; skew 500 ppm
I20260809 22:46:18.133651 1340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:46:18.136188 1340 webserver.cc:460] Webserver started at http://127.24.136.2:46447/ using document root <none> and password file <none>
I20260809 22:46:18.136950 1340 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:18.137058 1340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:18.137404 1340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:18.140615 1340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "f7ccfd121dae4f03bd5495934a249dfd"
format_stamp: "Formatted at 2026-08-09 22:46:18 on dist-test-slave-pww0"
server_key: "6a40f88743dd859d9b7cd7584195e44c"
server_key_iv: "997634c514c7efbd2b42aa9409b3632d"
server_key_version: "encryptionkey@0"
I20260809 22:46:18.141486 1340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "f7ccfd121dae4f03bd5495934a249dfd"
format_stamp: "Formatted at 2026-08-09 22:46:18 on dist-test-slave-pww0"
server_key: "6a40f88743dd859d9b7cd7584195e44c"
server_key_iv: "997634c514c7efbd2b42aa9409b3632d"
server_key_version: "encryptionkey@0"
I20260809 22:46:18.147179 1340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:46:18.151207 1359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:18.152772 1340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:18.152966 1340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "f7ccfd121dae4f03bd5495934a249dfd"
format_stamp: "Formatted at 2026-08-09 22:46:18 on dist-test-slave-pww0"
server_key: "6a40f88743dd859d9b7cd7584195e44c"
server_key_iv: "997634c514c7efbd2b42aa9409b3632d"
server_key_version: "encryptionkey@0"
I20260809 22:46:18.153208 1340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:18.159363 1340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:18.160347 1340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:18.160621 1340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:18.161615 1340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:18.163172 1340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:18.163285 1340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:18.163403 1340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:18.163491 1340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:18.196592 1340 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:45897
I20260809 22:46:18.196703 1558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:45897 every 8 connection(s)
I20260809 22:46:18.198036 1340 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:18.206928 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.200855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:51143 (local address 127.24.136.62:34583)
0809 22:46:18.201095 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:18.201103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:18.202621 (+ 1518us) server_negotiation.cc:407] Connection header received
0809 22:46:18.204628 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:18.204633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:18.204704 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:18.204799 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:18.206219 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:18.206224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:18.206227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:18.206248 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:18.206286 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:18.206300 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:18.206414 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:18.206634 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 22:46:18.206699 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:46:18.207782 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 1340
I20260809 22:46:18.207937 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:18.208019 1561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.201425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34583 (local address 127.24.136.2:51143)
0809 22:46:18.202437 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:18.202483 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:18.204315 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:18.205049 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:18.205058 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:18.205165 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:18.205821 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:18.205823 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:18.205865 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:18.205967 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:18.206392 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:18.206398 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:18.206403 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:46:18.206630 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 22:46:18.206885 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":320,"threads_started":1}
I20260809 22:46:18.208441 25120 external_mini_cluster.cc:1428] Setting key 406ad2ad69f7afb7b156fd726bbfce66
I20260809 22:46:18.209578 1559 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34583
I20260809 22:46:18.209895 1559 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:18.210527 1559 heartbeater.cc:511] Master 127.24.136.62:34583 requested a full tablet report, sending...
I20260809 22:46:18.211206 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34583
--builtin_ntp_servers=127.24.136.20:35141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 22:46:18.212131 983 ts_manager.cc:194] Registered new tserver with Master: f7ccfd121dae4f03bd5495934a249dfd (127.24.136.2:45897)
W20260809 22:46:18.562381 1562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:18.562682 1562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:18.562778 1562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:18.571059 1562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:18.571185 1562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:18.571358 1562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:18.581076 1562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:35141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:18.582490 1562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:18.584393 1562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:18.598093 1569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:18.598132 1571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:18.598160 1575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:18.599438 1562 server_base.cc:1034] running on GCE node
I20260809 22:46:18.600088 1562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:18.601084 1562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:18.602298 1562 hybrid_clock.cc:648] HybridClock initialized: now 1786315578602247 us; error 47 us; skew 500 ppm
I20260809 22:46:18.602617 1562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:46:18.605158 1562 webserver.cc:460] Webserver started at http://127.24.136.3:41419/ using document root <none> and password file <none>
I20260809 22:46:18.605865 1562 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:18.605957 1562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:18.606259 1562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:18.609256 1562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "0fe0609c52e647958cbe4bfbb54085f0"
format_stamp: "Formatted at 2026-08-09 22:46:18 on dist-test-slave-pww0"
server_key: "0b9e2643103629341784eb4bfa0764b6"
server_key_iv: "8336b5761a8fa94fb8b5c352a3cfc3bf"
server_key_version: "encryptionkey@0"
I20260809 22:46:18.610026 1562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "0fe0609c52e647958cbe4bfbb54085f0"
format_stamp: "Formatted at 2026-08-09 22:46:18 on dist-test-slave-pww0"
server_key: "0b9e2643103629341784eb4bfa0764b6"
server_key_iv: "8336b5761a8fa94fb8b5c352a3cfc3bf"
server_key_version: "encryptionkey@0"
I20260809 22:46:18.615388 1562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:46:18.619207 1583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:18.620656 1562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:18.620810 1562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "0fe0609c52e647958cbe4bfbb54085f0"
format_stamp: "Formatted at 2026-08-09 22:46:18 on dist-test-slave-pww0"
server_key: "0b9e2643103629341784eb4bfa0764b6"
server_key_iv: "8336b5761a8fa94fb8b5c352a3cfc3bf"
server_key_version: "encryptionkey@0"
I20260809 22:46:18.621035 1562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:18.626972 1562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:18.627823 1562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:18.628082 1562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:18.629019 1562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:18.630524 1562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:18.630615 1562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:18.630721 1562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:18.630787 1562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:18.663744 1562 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:39445
I20260809 22:46:18.663797 1767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:39445 every 8 connection(s)
I20260809 22:46:18.665197 1562 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:18.666564 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 1562
I20260809 22:46:18.666761 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:18.667377 25120 external_mini_cluster.cc:1428] Setting key 21b40c693a1c031e3daec161d02d4e9c
I20260809 22:46:18.674619 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.668650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:49755 (local address 127.24.136.62:34583)
0809 22:46:18.668879 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:18.668886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:18.670710 (+ 1824us) server_negotiation.cc:407] Connection header received
0809 22:46:18.672581 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:18.672585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:18.672655 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:18.672741 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:18.674056 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:18.674060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:18.674063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:18.674077 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:18.674105 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:18.674117 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:18.674214 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:18.674351 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 22:46:18.674405 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:46:18.675798 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.669184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34583 (local address 127.24.136.3:49755)
0809 22:46:18.670521 (+ 1337us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:18.670577 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:18.672287 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:18.672960 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:18.672970 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:18.673075 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:18.673767 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:18.673770 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:18.673811 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:18.673911 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:18.674188 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:18.674195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:18.674201 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:46:18.674423 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 22:46:18.674686 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":391,"threads_started":1}
I20260809 22:46:18.677234 1768 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34583
I20260809 22:46:18.677558 1768 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:18.678213 1768 heartbeater.cc:511] Master 127.24.136.62:34583 requested a full tablet report, sending...
I20260809 22:46:18.679898 983 ts_manager.cc:194] Registered new tserver with Master: 0fe0609c52e647958cbe4bfbb54085f0 (127.24.136.3:39445)
I20260809 22:46:18.683956 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:46:18.691718 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.688962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.24.136.62:34583)
0809 22:46:18.689149 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:18.689154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:18.689678 (+ 524us) server_negotiation.cc:407] Connection header received
0809 22:46:18.689845 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:18.689850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:18.689916 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:18.690032 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:18.691146 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:18.691151 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:18.691154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:18.691173 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:18.691211 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:18.691224 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:18.691350 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:18.691386 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 22:46:18.691439 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:46:18.696319 983 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:46:18.703706 983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:18.741182 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.718553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33207 (local address 127.0.0.1:51848)
0809 22:46:18.735025 (+ 16472us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:18.735053 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:18.735195 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:18.737953 (+ 2758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:18.737962 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:18.738068 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:18.739644 (+ 1576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:18.739648 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:18.739667 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:18.739710 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:18.740498 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:18.740514 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:18.740523 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:46:18.740701 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 22:46:18.740816 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16264,"mutex_wait_us":230,"thread_start_us":16488,"threads_started":1}
I20260809 22:46:18.741806 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.719953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.24.136.3:39445)
0809 22:46:18.739077 (+ 19124us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:18.739086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:18.739122 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:46:18.739195 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:18.739204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:18.739492 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:18.739677 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:18.740869 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:18.740879 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:18.740893 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:18.740914 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:18.740984 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:18.741005 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:18.741203 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:18.741306 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 22:46:18.741422 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18926,"thread_start_us":18743,"threads_started":1}
I20260809 22:46:18.742129 1798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.719394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.24.136.1:33207)
0809 22:46:18.736105 (+ 16711us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:18.736117 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:18.736170 (+ 53us) server_negotiation.cc:407] Connection header received
0809 22:46:18.736258 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:18.736267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:18.736577 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:18.736770 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:18.739984 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:18.739996 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:18.740014 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:18.740039 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:18.740221 (+ 182us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:18.740251 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:18.740434 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:18.741604 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0809 22:46:18.741723 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16486,"thread_start_us":16345,"threads_started":1}
I20260809 22:46:18.741712 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.719020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:39445 (local address 127.0.0.1:54014)
0809 22:46:18.736535 (+ 17515us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:18.736576 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:18.736719 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:18.739694 (+ 2975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:18.739699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:18.739723 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:18.740487 (+ 764us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:18.740491 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:18.740509 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:18.740551 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:18.741139 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:18.741145 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:18.741151 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:46:18.741292 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:46:18.741353 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17351,"thread_start_us":7904,"threads_started":1}
I20260809 22:46:18.744685 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.718839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.24.136.2:45897)
0809 22:46:18.737320 (+ 18481us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:18.737331 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:18.737373 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:46:18.737459 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:18.737469 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:18.737742 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:18.737930 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:18.742351 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:18.742361 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:18.742376 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:18.742409 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:18.742472 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:18.743371 (+ 899us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:18.744143 (+ 772us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:18.744245 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 22:46:18.744361 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18296,"thread_start_us":18157,"threads_started":1}
I20260809 22:46:18.745004 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:18.718553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:45897 (local address 127.0.0.1:38040)
0809 22:46:18.727592 (+ 9039us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:18.727650 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:18.727882 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:18.738349 (+ 10467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:18.738372 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:18.738397 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:18.739429 (+ 1032us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:18.739432 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:18.739497 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:18.742101 (+ 2604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:18.744089 (+ 1988us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:18.744095 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:18.744101 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:46:18.744247 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 22:46:18.744311 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8546,"thread_start_us":17623,"threads_started":1}
I20260809 22:46:18.747018 1216 tablet_service.cc:1467] Processing CreateTablet for tablet 47820dd3e54e47358adc7c517b69b3b6 (DEFAULT_TABLE table=test-table [id=c8b2a0886c5f47b2972f6e1b540499f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:18.747018 1670 tablet_service.cc:1467] Processing CreateTablet for tablet 47820dd3e54e47358adc7c517b69b3b6 (DEFAULT_TABLE table=test-table [id=c8b2a0886c5f47b2972f6e1b540499f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:18.748660 1670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47820dd3e54e47358adc7c517b69b3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:18.748675 1216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47820dd3e54e47358adc7c517b69b3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:18.749676 1442 tablet_service.cc:1467] Processing CreateTablet for tablet 47820dd3e54e47358adc7c517b69b3b6 (DEFAULT_TABLE table=test-table [id=c8b2a0886c5f47b2972f6e1b540499f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:18.751487 1442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47820dd3e54e47358adc7c517b69b3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:18.761230 1334 heartbeater.cc:503] Master 127.24.136.62:34583 was elected leader, sending a full tablet report...
I20260809 22:46:18.762625 1802 tablet_bootstrap.cc:492] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55: Bootstrap starting.
I20260809 22:46:18.766322 1802 tablet_bootstrap.cc:654] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:18.767829 1802 log.cc:824] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:18.770978 1802 tablet_bootstrap.cc:492] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55: No bootstrap required, opened a new log
I20260809 22:46:18.771274 1802 ts_tablet_manager.cc:1397] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 22:46:18.771756 1803 tablet_bootstrap.cc:492] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: Bootstrap starting.
I20260809 22:46:18.776333 1803 tablet_bootstrap.cc:654] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:18.776855 1807 tablet_bootstrap.cc:492] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: Bootstrap starting.
I20260809 22:46:18.778043 1803 log.cc:824] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:18.781080 1807 tablet_bootstrap.cc:654] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:18.780730 1802 raft_consensus.cc:348] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:18.781145 1803 tablet_bootstrap.cc:492] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: No bootstrap required, opened a new log
I20260809 22:46:18.781260 1802 raft_consensus.cc:374] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:18.781463 1803 ts_tablet_manager.cc:1397] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260809 22:46:18.781442 1802 raft_consensus.cc:729] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f87c6b3ed0f43d8be3aa81873c25c55, State: Initialized, Role: FOLLOWER
I20260809 22:46:18.782157 1802 consensus_queue.cc:260] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:18.782442 1807 log.cc:824] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:18.784219 1802 ts_tablet_manager.cc:1428] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260809 22:46:18.788524 1807 tablet_bootstrap.cc:492] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: No bootstrap required, opened a new log
I20260809 22:46:18.788844 1807 ts_tablet_manager.cc:1397] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260809 22:46:18.791226 1803 raft_consensus.cc:348] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:18.791715 1803 raft_consensus.cc:374] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:18.791827 1803 raft_consensus.cc:729] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ccfd121dae4f03bd5495934a249dfd, State: Initialized, Role: FOLLOWER
I20260809 22:46:18.792492 1803 consensus_queue.cc:260] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:18.793677 1559 heartbeater.cc:503] Master 127.24.136.62:34583 was elected leader, sending a full tablet report...
I20260809 22:46:18.794253 1803 ts_tablet_manager.cc:1428] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260809 22:46:18.797955 1807 raft_consensus.cc:348] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:18.798377 1807 raft_consensus.cc:374] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:18.798489 1807 raft_consensus.cc:729] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe0609c52e647958cbe4bfbb54085f0, State: Initialized, Role: FOLLOWER
I20260809 22:46:18.799226 1807 consensus_queue.cc:260] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:18.800382 1768 heartbeater.cc:503] Master 127.24.136.62:34583 was elected leader, sending a full tablet report...
I20260809 22:46:18.800899 1807 ts_tablet_manager.cc:1428] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260809 22:46:18.917135 1770 tablet.cc:2369] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:46:18.950228 1560 tablet.cc:2369] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:46:18.994467 1335 tablet.cc:2369] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:19.251598 1810 raft_consensus.cc:482] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:19.251929 1810 raft_consensus.cc:504] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:19.253650 1810 leader_election.cc:288] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7ccfd121dae4f03bd5495934a249dfd (127.24.136.2:45897), 0fe0609c52e647958cbe4bfbb54085f0 (127.24.136.3:39445)
I20260809 22:46:19.261379 1812 raft_consensus.cc:482] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:19.261780 1812 raft_consensus.cc:504] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:19.268306 1812 leader_election.cc:288] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f87c6b3ed0f43d8be3aa81873c25c55 (127.24.136.1:33207), 0fe0609c52e647958cbe4bfbb54085f0 (127.24.136.3:39445)
I20260809 22:46:19.270787 1818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.254150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:54801 (local address 127.24.136.3:39445)
0809 22:46:19.267842 (+ 13692us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:19.267848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:19.267867 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:46:19.267930 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:19.267936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:19.267994 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:19.268109 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:19.269881 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:19.269886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:19.269892 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:19.269913 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:19.269969 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:19.269986 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:19.270084 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:19.270360 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 22:46:19.270433 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13568,"thread_start_us":13628,"threads_started":1}
I20260809 22:46:19.271413 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.254185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:39445 (local address 127.24.136.1:54801)
0809 22:46:19.267601 (+ 13416us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.267623 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:19.267726 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:19.268343 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:19.268348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:19.268375 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:19.269591 (+ 1216us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:19.269595 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:19.269617 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:19.269692 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:19.270174 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:19.270179 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:19.270183 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 22:46:19.270926 (+ 743us) client_negotiation.cc:240] Negotiation successful
0809 22:46:19.270997 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13280,"thread_start_us":13008,"threads_started":1}
I20260809 22:46:19.273049 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.266236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:46221 (local address 127.24.136.1:33207)
0809 22:46:19.269937 (+ 3701us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:19.269942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:19.269963 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:46:19.270006 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:19.270010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:19.270059 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:19.270160 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:19.271964 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:19.271970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:19.271974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:19.271996 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:19.272052 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:19.272069 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:19.272218 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:19.272421 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 22:46:19.272485 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3583,"thread_start_us":3475,"threads_started":1}
I20260809 22:46:19.273093 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.266610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:34445 (local address 127.24.136.3:39445)
0809 22:46:19.269194 (+ 2584us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:19.269200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:19.269221 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:46:19.269282 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:19.269287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:19.269345 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:19.270431 (+ 1086us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:19.272026 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:19.272032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:19.272037 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:19.272063 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:19.272113 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:19.272130 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:19.272241 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:19.272481 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 22:46:19.272544 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2453,"thread_start_us":2569,"threads_started":1}
I20260809 22:46:19.273216 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.254144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:45897 (local address 127.24.136.1:36239)
0809 22:46:19.266774 (+ 12630us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.266799 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:19.266945 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:19.271347 (+ 4402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:19.271352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:19.271384 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:19.272096 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:19.272099 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:19.272115 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:19.272154 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:19.272547 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:19.272552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:19.272557 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:46:19.272694 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 22:46:19.272748 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12454,"spinlock_wait_cycles":32128,"thread_start_us":14395,"threads_started":1}
I20260809 22:46:19.273222 1699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47820dd3e54e47358adc7c517b69b3b6" candidate_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe0609c52e647958cbe4bfbb54085f0" is_pre_election: true
I20260809 22:46:19.273815 1699 raft_consensus.cc:2454] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f87c6b3ed0f43d8be3aa81873c25c55 in term 0.
I20260809 22:46:19.274142 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.254145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:36239 (local address 127.24.136.2:45897)
0809 22:46:19.271005 (+ 16860us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:19.271011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:19.271031 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:19.271080 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:19.271084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:19.271139 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:19.271318 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:19.272308 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:19.272313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:19.272317 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:19.272336 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:19.272382 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:19.272397 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:19.272495 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:19.273552 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0809 22:46:19.273620 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16722,"thread_start_us":17620,"threads_started":1}
I20260809 22:46:19.274024 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.264813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:39445 (local address 127.24.136.2:34445)
0809 22:46:19.265473 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.265494 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:19.265637 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:19.270453 (+ 4816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:19.270458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:19.270479 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:19.271510 (+ 1031us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:19.271513 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:19.271539 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:19.271610 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:19.272344 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:19.272349 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:19.272353 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 22:46:19.273412 (+ 1059us) client_negotiation.cc:240] Negotiation successful
0809 22:46:19.273480 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":3097,"threads_started":1}
I20260809 22:46:19.274544 1132 leader_election.cc:302] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fe0609c52e647958cbe4bfbb54085f0, 1f87c6b3ed0f43d8be3aa81873c25c55; no voters:
I20260809 22:46:19.274449 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.265494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33207 (local address 127.24.136.2:46221)
0809 22:46:19.269232 (+ 3738us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.269252 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:19.269355 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:19.270238 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:19.270245 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:19.270289 (+ 44us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:19.271734 (+ 1445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:46:19.271737 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:19.271753 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:46:19.271798 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:19.272275 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:19.272281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:19.272286 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:46:19.273828 (+ 1542us) client_negotiation.cc:240] Negotiation successful
0809 22:46:19.273889 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3605,"thread_start_us":3414,"threads_started":1}
I20260809 22:46:19.275060 1810 raft_consensus.cc:2781] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:19.275220 1810 raft_consensus.cc:482] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:19.275347 1810 raft_consensus.cc:3037] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:19.275255 1479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47820dd3e54e47358adc7c517b69b3b6" candidate_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ccfd121dae4f03bd5495934a249dfd" is_pre_election: true
I20260809 22:46:19.275545 1699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47820dd3e54e47358adc7c517b69b3b6" candidate_uuid: "f7ccfd121dae4f03bd5495934a249dfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe0609c52e647958cbe4bfbb54085f0" is_pre_election: true
I20260809 22:46:19.275763 1699 raft_consensus.cc:2454] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ccfd121dae4f03bd5495934a249dfd in term 0.
I20260809 22:46:19.275753 1479 raft_consensus.cc:2454] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f87c6b3ed0f43d8be3aa81873c25c55 in term 0.
I20260809 22:46:19.276306 1361 leader_election.cc:302] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fe0609c52e647958cbe4bfbb54085f0, f7ccfd121dae4f03bd5495934a249dfd; no voters:
I20260809 22:46:19.276376 1264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47820dd3e54e47358adc7c517b69b3b6" candidate_uuid: "f7ccfd121dae4f03bd5495934a249dfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" is_pre_election: true
I20260809 22:46:19.276696 1812 raft_consensus.cc:2781] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:19.276803 1812 raft_consensus.cc:482] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:19.276888 1812 raft_consensus.cc:3037] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:19.278270 1810 raft_consensus.cc:504] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:19.278745 1264 raft_consensus.cc:2379] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f7ccfd121dae4f03bd5495934a249dfd in current term 1: Already voted for candidate 1f87c6b3ed0f43d8be3aa81873c25c55 in this term.
I20260809 22:46:19.279047 1810 leader_election.cc:288] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [CANDIDATE]: Term 1 election: Requested vote from peers f7ccfd121dae4f03bd5495934a249dfd (127.24.136.2:45897), 0fe0609c52e647958cbe4bfbb54085f0 (127.24.136.3:39445)
I20260809 22:46:19.279199 1812 raft_consensus.cc:504] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:19.279690 1479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47820dd3e54e47358adc7c517b69b3b6" candidate_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ccfd121dae4f03bd5495934a249dfd"
I20260809 22:46:19.280000 1479 raft_consensus.cc:2379] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f87c6b3ed0f43d8be3aa81873c25c55 in current term 1: Already voted for candidate f7ccfd121dae4f03bd5495934a249dfd in this term.
I20260809 22:46:19.280112 1699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47820dd3e54e47358adc7c517b69b3b6" candidate_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe0609c52e647958cbe4bfbb54085f0"
I20260809 22:46:19.280340 1699 raft_consensus.cc:3037] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:19.280972 1812 leader_election.cc:288] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [CANDIDATE]: Term 1 election: Requested vote from peers 1f87c6b3ed0f43d8be3aa81873c25c55 (127.24.136.1:33207), 0fe0609c52e647958cbe4bfbb54085f0 (127.24.136.3:39445)
I20260809 22:46:19.281106 1698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47820dd3e54e47358adc7c517b69b3b6" candidate_uuid: "f7ccfd121dae4f03bd5495934a249dfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe0609c52e647958cbe4bfbb54085f0"
I20260809 22:46:19.281478 1264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47820dd3e54e47358adc7c517b69b3b6" candidate_uuid: "f7ccfd121dae4f03bd5495934a249dfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55"
I20260809 22:46:19.281736 1264 raft_consensus.cc:2379] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f7ccfd121dae4f03bd5495934a249dfd in current term 1: Already voted for candidate 1f87c6b3ed0f43d8be3aa81873c25c55 in this term.
I20260809 22:46:19.282156 1361 leader_election.cc:302] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7ccfd121dae4f03bd5495934a249dfd; no voters: 0fe0609c52e647958cbe4bfbb54085f0, 1f87c6b3ed0f43d8be3aa81873c25c55
I20260809 22:46:19.282457 1812 raft_consensus.cc:2726] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 22:46:19.283444 1699 raft_consensus.cc:2454] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f87c6b3ed0f43d8be3aa81873c25c55 in term 1.
I20260809 22:46:19.283890 1132 leader_election.cc:302] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0fe0609c52e647958cbe4bfbb54085f0, 1f87c6b3ed0f43d8be3aa81873c25c55; no voters: f7ccfd121dae4f03bd5495934a249dfd
I20260809 22:46:19.284209 1810 raft_consensus.cc:2781] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:19.284680 1810 raft_consensus.cc:686] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 1 LEADER]: Becoming Leader. State: Replica: 1f87c6b3ed0f43d8be3aa81873c25c55, State: Running, Role: LEADER
I20260809 22:46:19.285372 1810 consensus_queue.cc:237] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } }
I20260809 22:46:19.289839 982 catalog_manager.cc:5495] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f87c6b3ed0f43d8be3aa81873c25c55 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f87c6b3ed0f43d8be3aa81873c25c55" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 } health_report { overall_health: UNKNOWN } } }
I20260809 22:46:19.321725 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.318939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.24.136.1:33207)
0809 22:46:19.319171 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:19.319178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:19.319750 (+ 572us) server_negotiation.cc:407] Connection header received
0809 22:46:19.319992 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:19.319996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:19.320102 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:19.320194 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:19.321129 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:19.321135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:19.321138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:19.321159 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:19.321200 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:19.321213 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:19.321320 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:19.321390 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 22:46:19.321445 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:46:19.330659 1479 raft_consensus.cc:1261] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 1 FOLLOWER]: Refusing update from remote peer 1f87c6b3ed0f43d8be3aa81873c25c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:19.330637 1699 raft_consensus.cc:1261] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 1 FOLLOWER]: Refusing update from remote peer 1f87c6b3ed0f43d8be3aa81873c25c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:19.331759 1832 consensus_queue.cc:1035] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7ccfd121dae4f03bd5495934a249dfd" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:19.332165 1810 consensus_queue.cc:1035] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fe0609c52e647958cbe4bfbb54085f0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:19.359609 983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:19.359929 983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:19.364164 983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ec65745fad1410dbebce6e8b29d0c62: Sending DeleteTablet for 3 replicas of tablet 47820dd3e54e47358adc7c517b69b3b6
I20260809 22:46:19.365581 1670 tablet_service.cc:1514] Processing DeleteTablet for tablet 47820dd3e54e47358adc7c517b69b3b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:19 UTC) from {username='slave'} at 127.0.0.1:54014
I20260809 22:46:19.365556 1216 tablet_service.cc:1514] Processing DeleteTablet for tablet 47820dd3e54e47358adc7c517b69b3b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:19 UTC) from {username='slave'} at 127.0.0.1:51848
I20260809 22:46:19.365937 1442 tablet_service.cc:1514] Processing DeleteTablet for tablet 47820dd3e54e47358adc7c517b69b3b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:19 UTC) from {username='slave'} at 127.0.0.1:38040
I20260809 22:46:19.368129 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 1100
I20260809 22:46:19.378604 1867 tablet_replica.cc:330] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55: stopping tablet replica
I20260809 22:46:19.379323 1868 tablet_replica.cc:330] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: stopping tablet replica
I20260809 22:46:19.379755 1865 tablet_replica.cc:330] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: stopping tablet replica
I20260809 22:46:19.380198 1865 raft_consensus.cc:2229] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:19.380569 1868 raft_consensus.cc:2229] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:19.380844 1865 raft_consensus.cc:2258] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:19.380239 1867 raft_consensus.cc:2229] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:19.381695 1867 raft_consensus.cc:2258] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:19.382136 1868 raft_consensus.cc:2258] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:19.383351 1865 ts_tablet_manager.cc:1905] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:19.384538 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.369587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51870 (local address 127.24.136.1:33207)
0809 22:46:19.369776 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:19.369783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:19.381828 (+ 12045us) server_negotiation.cc:407] Connection header received
0809 22:46:19.382170 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:19.382176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:19.382250 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:19.382335 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:19.383646 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:19.383652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:19.383657 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:19.383681 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:19.383723 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:19.383739 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:19.383859 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:19.384136 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 22:46:19.384214 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:46:19.385713 1328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:19.386147 1868 ts_tablet_manager.cc:1905] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:19.391585 1865 ts_tablet_manager.cc:1918] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:19.391778 1865 log.cc:1196] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/47820dd3e54e47358adc7c517b69b3b6
I20260809 22:46:19.392346 1865 ts_tablet_manager.cc:1939] T 47820dd3e54e47358adc7c517b69b3b6 P 0fe0609c52e647958cbe4bfbb54085f0: Deleting consensus metadata
I20260809 22:46:19.393618 1868 ts_tablet_manager.cc:1918] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:19.393800 1868 log.cc:1196] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/47820dd3e54e47358adc7c517b69b3b6
I20260809 22:46:19.394331 1868 ts_tablet_manager.cc:1939] T 47820dd3e54e47358adc7c517b69b3b6 P f7ccfd121dae4f03bd5495934a249dfd: Deleting consensus metadata
I20260809 22:46:19.394284 953 catalog_manager.cc:4841] TS 0fe0609c52e647958cbe4bfbb54085f0 (127.24.136.3:39445): tablet 47820dd3e54e47358adc7c517b69b3b6 (table test-table [id=c8b2a0886c5f47b2972f6e1b540499f5]) successfully deleted
I20260809 22:46:19.396486 954 catalog_manager.cc:4841] TS f7ccfd121dae4f03bd5495934a249dfd (127.24.136.2:45897): tablet 47820dd3e54e47358adc7c517b69b3b6 (table test-table [id=c8b2a0886c5f47b2972f6e1b540499f5]) successfully deleted
I20260809 22:46:19.386664 1867 ts_tablet_manager.cc:1905] T 47820dd3e54e47358adc7c517b69b3b6 P 1f87c6b3ed0f43d8be3aa81873c25c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:19.483620 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 1100
I20260809 22:46:19.508478 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 1340
W20260809 22:46:19.508484 957 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.136.1:33207 (error 108)
I20260809 22:46:19.510823 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.509845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33207 (local address 127.0.0.1:51872)
0809 22:46:19.510405 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.510532 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":241,"threads_started":1}
W20260809 22:46:19.511087 957 catalog_manager.cc:4584] TS 1f87c6b3ed0f43d8be3aa81873c25c55 (127.24.136.1:33207): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47820dd3e54e47358adc7c517b69b3b6: Network error: Client connection negotiation failed: client connection to 127.24.136.1:33207: connect: Connection refused (error 111)
I20260809 22:46:19.511919 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.509272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.24.136.2:45897)
0809 22:46:19.509473 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:19.509480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:19.509702 (+ 222us) server_negotiation.cc:407] Connection header received
0809 22:46:19.509951 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:19.509957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:19.510066 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:19.510192 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:19.511293 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:19.511298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:19.511303 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:19.511325 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:19.511367 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:19.511386 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:19.511552 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:19.511683 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 22:46:19.511740 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:46:19.512596 1550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:19.564917 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.564439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33207 (local address 127.0.0.1:51884)
0809 22:46:19.564675 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.564771 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 22:46:19.566068 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.565711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33207 (local address 127.0.0.1:51896)
0809 22:46:19.565844 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.565929 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 22:46:19.604660 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 1340
I20260809 22:46:19.626688 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.626125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33207 (local address 127.0.0.1:51900)
0809 22:46:19.626369 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.626487 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:46:19.628068 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.627573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33207 (local address 127.0.0.1:51916)
0809 22:46:19.627752 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.627857 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 22:46:19.637230 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 1562
I20260809 22:46:19.640611 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.638085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.24.136.3:39445)
0809 22:46:19.638321 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:19.638329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:19.638415 (+ 86us) server_negotiation.cc:407] Connection header received
0809 22:46:19.638643 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:19.638649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:19.638752 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:19.638857 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:19.639884 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:19.639890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:19.639893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:19.639913 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:19.639950 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:19.639965 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:19.640128 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:19.640226 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 22:46:19.640301 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 22:46:19.641444 1762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:19.709067 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.708505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33207 (local address 127.0.0.1:51928)
0809 22:46:19.708815 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.708914 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 22:46:19.710245 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:19.709782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33207 (local address 127.0.0.1:51942)
0809 22:46:19.710013 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:19.710092 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260809 22:46:19.729873 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 1562
I20260809 22:46:19.756381 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 931
I20260809 22:46:19.757721 1041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:19.866542 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 931
2026-08-09T22:46:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3233 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 22:46:19.918591 25120 mini_oidc.cc:152] Starting JWKS server
I20260809 22:46:19.921883 25120 webserver.cc:460] Webserver started at https://127.0.0.1:37097/ using document root <none> and password file <none>
I20260809 22:46:19.922284 25120 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:46:19.923012 25120 webserver.cc:460] Webserver started at http://127.0.0.1:33755/ using document root <none> and password file <none>
2026-08-09T22:46:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:19Z Disabled control of system clock
I20260809 22:46:19.959528 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45517
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:45205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45517
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37097/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/testchainca.cert with env {}
W20260809 22:46:20.311266 1891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:20.311681 1891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:20.311825 1891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:20.319402 1891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:20.319602 1891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:20.319676 1891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:20.319762 1891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:20.319828 1891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:46:20.319913 1891 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:46:20.319964 1891 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37097/jwks.json
I20260809 22:46:20.331024 1891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45517
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37097/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:20.332598 1891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:20.334033 1891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:20.346508 1897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:20.346441 1900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:20.347186 1898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:20.347419 1891 server_base.cc:1034] running on GCE node
I20260809 22:46:20.348333 1891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:20.349856 1891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:20.351087 1891 hybrid_clock.cc:648] HybridClock initialized: now 1786315580351015 us; error 55 us; skew 500 ppm
I20260809 22:46:20.351497 1891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:46:20.358973 1891 webserver.cc:460] Webserver started at http://127.24.136.62:46853/ using document root <none> and password file <none>
I20260809 22:46:20.359853 1891 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:20.359982 1891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:20.360316 1891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:20.363516 1891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "1320948184b4471985be23759686381d"
format_stamp: "Formatted at 2026-08-09 22:46:20 on dist-test-slave-pww0"
server_key: "e2f6f9087ef8c659b3e8b4c3ab0fc465"
server_key_iv: "baf0745825e46d162edadcb627704cb6"
server_key_version: "encryptionkey@0"
I20260809 22:46:20.364430 1891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "1320948184b4471985be23759686381d"
format_stamp: "Formatted at 2026-08-09 22:46:20 on dist-test-slave-pww0"
server_key: "e2f6f9087ef8c659b3e8b4c3ab0fc465"
server_key_iv: "baf0745825e46d162edadcb627704cb6"
server_key_version: "encryptionkey@0"
I20260809 22:46:20.370085 1891 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:46:20.374006 1910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:20.375573 1891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:46:20.375773 1891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "1320948184b4471985be23759686381d"
format_stamp: "Formatted at 2026-08-09 22:46:20 on dist-test-slave-pww0"
server_key: "e2f6f9087ef8c659b3e8b4c3ab0fc465"
server_key_iv: "baf0745825e46d162edadcb627704cb6"
server_key_version: "encryptionkey@0"
I20260809 22:46:20.376004 1891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:20.388705 1891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:20.389648 1891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:20.389973 1891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:20.408919 1891 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:45517
I20260809 22:46:20.408939 1993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:45517 every 8 connection(s)
I20260809 22:46:20.410367 1891 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:20.414327 1998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:20.414665 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 1891
I20260809 22:46:20.414865 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:20.415537 25120 external_mini_cluster.cc:1428] Setting key c8dcd32254d2ec7399c29ee98125ee4f
I20260809 22:46:20.428293 1998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d: Bootstrap starting.
I20260809 22:46:20.430011 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:20.417547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45094 (local address 127.24.136.62:45517)
0809 22:46:20.418667 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:20.418681 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:20.418733 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:46:20.420488 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:20.420515 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:20.421060 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:20.421445 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:20.422107 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:20.423404 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:20.425385 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:20.426130 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:20.426288 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:20.427612 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:20.427620 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:20.427636 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:20.428195 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:20.428291 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:20.428320 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:20.428428 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:20.428650 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 22:46:20.428987 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":390,"threads_started":1}
I20260809 22:46:20.431752 1998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:20.432861 1998 log.cc:824] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:20.433972 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:46:20.435829 1998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d: No bootstrap required, opened a new log
I20260809 22:46:20.443671 1998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320948184b4471985be23759686381d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45517 } }
I20260809 22:46:20.444061 1998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:20.444169 1998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1320948184b4471985be23759686381d, State: Initialized, Role: FOLLOWER
I20260809 22:46:20.444784 1998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320948184b4471985be23759686381d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45517 } }
I20260809 22:46:20.445003 1998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:20.445143 1998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:20.445302 1998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:20.447749 1998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320948184b4471985be23759686381d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45517 } }
I20260809 22:46:20.448323 1998 leader_election.cc:302] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1320948184b4471985be23759686381d; no voters:
I20260809 22:46:20.448907 1998 leader_election.cc:288] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:20.449051 2012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:20.449786 2012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [term 1 LEADER]: Becoming Leader. State: Replica: 1320948184b4471985be23759686381d, State: Running, Role: LEADER
I20260809 22:46:20.450316 2012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320948184b4471985be23759686381d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45517 } }
I20260809 22:46:20.450968 1998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:20.453019 2014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1320948184b4471985be23759686381d. Latest consensus state: current_term: 1 leader_uuid: "1320948184b4471985be23759686381d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320948184b4471985be23759686381d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45517 } } }
I20260809 22:46:20.453063 2013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1320948184b4471985be23759686381d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1320948184b4471985be23759686381d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45517 } } }
I20260809 22:46:20.453497 2013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:20.453497 2014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:20.458417 2022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:20.463910 2022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:20.474567 2022 catalog_manager.cc:1347] Generated new cluster ID: 7ec09df0cc604badad0244f32c087701
I20260809 22:46:20.474711 2022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:20.493937 2022 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:20.495368 2022 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:20.506400 2022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1320948184b4471985be23759686381d: Generated new TSK 0
I20260809 22:46:20.507718 2022 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 22:46:20.564003 2051 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 22:46:20.564287 2051 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:46:20.565338 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:20.536285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.24.136.62:45517)
0809 22:46:20.536513 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:20.536519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:20.537200 (+ 681us) server_negotiation.cc:407] Connection header received
0809 22:46:20.537667 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:20.537671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:20.537775 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:20.537914 (+ 139us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:46:20.538638 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:20.539381 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:20.541540 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:20.542030 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:20.542141 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:20.543031 (+ 890us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:46:20.564753 (+ 21722us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 22:46:20.565096 (+ 343us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45108: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 22:46:20.565460 2002 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45108: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 22:46:20.567512 2049 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:20.536146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45517 (local address 127.0.0.1:45108)
0809 22:46:20.537062 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:20.537085 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:20.537329 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:20.537977 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:20.537983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:20.537998 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:46:20.538431 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:20.538446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:20.539563 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:20.539570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:20.541365 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:20.541384 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:20.542210 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:20.542217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:20.542548 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:20.542649 (+ 101us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:46:20.565108 (+ 22459us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 22:46:20.565519 (+ 411us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.136.62:45517: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":503,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.136.62:45517: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37097/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 22:46:20.569420 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 1891
I20260809 22:46:20.570864 1986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:20.669847 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 1891
2026-08-09T22:46:20Z chronyd exiting
I20260809 22:46:20.729713 25120 test_util.cc:175] -----------------------------------------------
I20260809 22:46:20.729882 25120 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315533505708-25120-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (816 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 22:46:20.733908 25120 mini_oidc.cc:152] Starting JWKS server
I20260809 22:46:20.735859 25120 webserver.cc:460] Webserver started at https://127.0.0.1:40247/ using document root <none> and password file <none>
I20260809 22:46:20.736112 25120 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:46:20.736850 25120 webserver.cc:460] Webserver started at http://127.0.0.1:45607/ using document root <none> and password file <none>
2026-08-09T22:46:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:20Z Disabled control of system clock
I20260809 22:46:20.772898 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:46501
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:37541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:46501
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40247/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/testchainca.cert with env {}
W20260809 22:46:21.135582 2061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:21.135937 2061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:21.136070 2061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:21.143713 2061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:21.143896 2061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:21.143958 2061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:21.144032 2061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:21.144090 2061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:46:21.144174 2061 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:46:21.144244 2061 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40247/jwks.json
I20260809 22:46:21.155258 2061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:46501
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:46501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40247/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:21.156695 2061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:21.158042 2061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:21.170696 2070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:21.170795 2068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:21.170756 2072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:21.171350 2061 server_base.cc:1034] running on GCE node
I20260809 22:46:21.172305 2061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:21.173964 2061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:21.175220 2061 hybrid_clock.cc:648] HybridClock initialized: now 1786315581175158 us; error 49 us; skew 500 ppm
I20260809 22:46:21.175674 2061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:46:21.178280 2061 webserver.cc:460] Webserver started at http://127.24.136.62:37641/ using document root <none> and password file <none>
I20260809 22:46:21.179085 2061 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:21.179207 2061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:21.179608 2061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:21.182787 2061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "c4685c13d22d4de5ad86ce651701f26a"
format_stamp: "Formatted at 2026-08-09 22:46:21 on dist-test-slave-pww0"
server_key: "585b42e58f2dfaa30ef20d06d6232c15"
server_key_iv: "1ace1833d448a10ed30a3e31ba60f4e8"
server_key_version: "encryptionkey@0"
I20260809 22:46:21.183782 2061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "c4685c13d22d4de5ad86ce651701f26a"
format_stamp: "Formatted at 2026-08-09 22:46:21 on dist-test-slave-pww0"
server_key: "585b42e58f2dfaa30ef20d06d6232c15"
server_key_iv: "1ace1833d448a10ed30a3e31ba60f4e8"
server_key_version: "encryptionkey@0"
I20260809 22:46:21.189570 2061 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:46:21.193686 2086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:21.195742 2061 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:46:21.195989 2061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "c4685c13d22d4de5ad86ce651701f26a"
format_stamp: "Formatted at 2026-08-09 22:46:21 on dist-test-slave-pww0"
server_key: "585b42e58f2dfaa30ef20d06d6232c15"
server_key_iv: "1ace1833d448a10ed30a3e31ba60f4e8"
server_key_version: "encryptionkey@0"
I20260809 22:46:21.196267 2061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:21.223091 2061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:21.224126 2061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:21.224473 2061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:21.243424 2061 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:46501
I20260809 22:46:21.243428 2179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:46501 every 8 connection(s)
I20260809 22:46:21.244992 2061 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:21.248844 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 2061
I20260809 22:46:21.249044 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:21.249155 2181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:21.249734 25120 external_mini_cluster.cc:1428] Setting key 727168cfa507d08924d8272cfc09063f
I20260809 22:46:21.266166 2181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a: Bootstrap starting.
I20260809 22:46:21.266654 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:21.251760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57608 (local address 127.24.136.62:46501)
0809 22:46:21.253085 (+ 1325us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:21.253116 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:21.253188 (+ 72us) server_negotiation.cc:407] Connection header received
0809 22:46:21.254939 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:21.254975 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:21.255889 (+ 914us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:21.256454 (+ 565us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:21.257094 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:21.259186 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:21.260753 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:21.261867 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:21.262084 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:21.263528 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:21.263547 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:21.263572 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:21.264371 (+ 799us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:21.264460 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:21.264492 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:21.264617 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:21.264876 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 22:46:21.265266 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":466,"threads_started":1}
I20260809 22:46:21.270290 2181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:21.271806 2181 log.cc:824] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:21.273248 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:46:21.278585 2181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a: No bootstrap required, opened a new log
I20260809 22:46:21.286641 2181 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4685c13d22d4de5ad86ce651701f26a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46501 } }
I20260809 22:46:21.287046 2181 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:21.287150 2181 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4685c13d22d4de5ad86ce651701f26a, State: Initialized, Role: FOLLOWER
I20260809 22:46:21.287787 2181 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4685c13d22d4de5ad86ce651701f26a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46501 } }
I20260809 22:46:21.288022 2181 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:21.288154 2181 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:21.288308 2181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:21.290591 2181 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4685c13d22d4de5ad86ce651701f26a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46501 } }
I20260809 22:46:21.291185 2181 leader_election.cc:302] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4685c13d22d4de5ad86ce651701f26a; no voters:
I20260809 22:46:21.291874 2181 leader_election.cc:288] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:21.292039 2188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:21.292780 2188 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [term 1 LEADER]: Becoming Leader. State: Replica: c4685c13d22d4de5ad86ce651701f26a, State: Running, Role: LEADER
I20260809 22:46:21.293385 2188 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4685c13d22d4de5ad86ce651701f26a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46501 } }
I20260809 22:46:21.294016 2181 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:21.296468 2193 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4685c13d22d4de5ad86ce651701f26a. Latest consensus state: current_term: 1 leader_uuid: "c4685c13d22d4de5ad86ce651701f26a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4685c13d22d4de5ad86ce651701f26a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46501 } } }
I20260809 22:46:21.296950 2193 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:21.296509 2191 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4685c13d22d4de5ad86ce651701f26a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4685c13d22d4de5ad86ce651701f26a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 46501 } } }
I20260809 22:46:21.297637 2191 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:21.297972 2205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:21.304271 2205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:21.314294 2205 catalog_manager.cc:1347] Generated new cluster ID: 4ea880335a1d495597e85b77dc600597
I20260809 22:46:21.314435 2205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:21.328781 2205 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:21.330802 2205 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:21.340745 2205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4685c13d22d4de5ad86ce651701f26a: Generated new TSK 0
I20260809 22:46:21.341977 2205 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 22:46:21.376994 2240 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 22:46:21.377115 2240 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:46:21.378728 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:21.356542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57618 (local address 127.24.136.62:46501)
0809 22:46:21.356769 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:21.356776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:21.357259 (+ 483us) server_negotiation.cc:407] Connection header received
0809 22:46:21.357478 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:21.357484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:21.357588 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:21.357799 (+ 211us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:46:21.358241 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:21.359382 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:21.360845 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:21.361571 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:21.361709 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:21.362304 (+ 595us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:46:21.377878 (+ 15574us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 22:46:21.378373 (+ 495us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57618: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 22:46:21.378873 2184 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57618: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 22:46:21.378885 2238 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:21.356417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:46501 (local address 127.0.0.1:57618)
0809 22:46:21.357127 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:21.357149 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:21.357293 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:21.357801 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:21.357805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:21.357814 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:46:21.358087 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:21.358095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:21.359579 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:21.359584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:21.360683 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:21.360698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:21.361788 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:21.361793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:21.361942 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:21.361982 (+ 40us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:46:21.378247 (+ 16265us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 22:46:21.378512 (+ 265us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.136.62:46501: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":389,"threads_started":1}
I20260809 22:46:21.380414 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 2061
I20260809 22:46:21.381762 2172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:21.474872 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 2061
2026-08-09T22:46:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (805 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 22:46:21.539223 25120 mini_oidc.cc:152] Starting JWKS server
I20260809 22:46:21.541265 25120 webserver.cc:460] Webserver started at https://127.0.0.1:46825/ using document root <none> and password file <none>
I20260809 22:46:21.541548 25120 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:46:21.542245 25120 webserver.cc:460] Webserver started at http://127.0.0.1:38611/ using document root <none> and password file <none>
2026-08-09T22:46:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:21Z Disabled control of system clock
I20260809 22:46:21.576746 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38395
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38395
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46825/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 22:46:21.927193 2253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:21.927599 2253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:21.927742 2253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:21.935267 2253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:21.935396 2253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:21.935439 2253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:21.935546 2253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:21.935601 2253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:46:21.935689 2253 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:46:21.935734 2253 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46825/jwks.json
I20260809 22:46:21.945644 2253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38395
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46825/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:21.947094 2253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:21.949112 2253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:21.961971 2261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:21.961923 2267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:21.961994 2262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:21.962898 2253 server_base.cc:1034] running on GCE node
I20260809 22:46:21.963833 2253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:21.965437 2253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:21.966653 2253 hybrid_clock.cc:648] HybridClock initialized: now 1786315581966599 us; error 47 us; skew 500 ppm
I20260809 22:46:21.966996 2253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:46:21.969429 2253 webserver.cc:460] Webserver started at http://127.24.136.62:35119/ using document root <none> and password file <none>
I20260809 22:46:21.970177 2253 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:21.970288 2253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:21.970594 2253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:21.973789 2253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "7f8180b3bb26479fb6175f9a7e2f9333"
format_stamp: "Formatted at 2026-08-09 22:46:21 on dist-test-slave-pww0"
server_key: "eca77dc7aeef85b07c8f9da8ad03fa4d"
server_key_iv: "0493a70362ad436889ca3f18559701a2"
server_key_version: "encryptionkey@0"
I20260809 22:46:21.974681 2253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "7f8180b3bb26479fb6175f9a7e2f9333"
format_stamp: "Formatted at 2026-08-09 22:46:21 on dist-test-slave-pww0"
server_key: "eca77dc7aeef85b07c8f9da8ad03fa4d"
server_key_iv: "0493a70362ad436889ca3f18559701a2"
server_key_version: "encryptionkey@0"
I20260809 22:46:21.980476 2253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:46:21.984449 2277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:21.986258 2253 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:46:21.986447 2253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "7f8180b3bb26479fb6175f9a7e2f9333"
format_stamp: "Formatted at 2026-08-09 22:46:21 on dist-test-slave-pww0"
server_key: "eca77dc7aeef85b07c8f9da8ad03fa4d"
server_key_iv: "0493a70362ad436889ca3f18559701a2"
server_key_version: "encryptionkey@0"
I20260809 22:46:21.986675 2253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:22.001359 2253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:22.002269 2253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:22.002588 2253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:22.021142 2253 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38395
I20260809 22:46:22.021147 2375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38395 every 8 connection(s)
I20260809 22:46:22.022681 2253 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:22.026759 2377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:22.032696 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 2253
I20260809 22:46:22.032927 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:22.033563 25120 external_mini_cluster.cc:1428] Setting key c68d57ed84c5af9a56a5b7828729d067
I20260809 22:46:22.038534 2377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333: Bootstrap starting.
I20260809 22:46:22.042001 2377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:22.043120 2377 log.cc:824] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:22.046276 2377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333: No bootstrap required, opened a new log
I20260809 22:46:22.047847 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:22.035689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46406 (local address 127.24.136.62:38395)
0809 22:46:22.036805 (+ 1116us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:22.036818 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:22.036868 (+ 50us) server_negotiation.cc:407] Connection header received
0809 22:46:22.038586 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:22.038611 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:22.039173 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:22.039609 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:22.040251 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.041584 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.043033 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.043900 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.044110 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:22.045287 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:22.045295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:22.045316 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:22.045908 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:22.046002 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:22.046027 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:22.046145 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:22.046389 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 22:46:22.046726 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":351,"threads_started":1}
I20260809 22:46:22.052102 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:46:22.054639 2377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8180b3bb26479fb6175f9a7e2f9333" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38395 } }
I20260809 22:46:22.055068 2377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:22.055171 2377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8180b3bb26479fb6175f9a7e2f9333, State: Initialized, Role: FOLLOWER
I20260809 22:46:22.055845 2377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8180b3bb26479fb6175f9a7e2f9333" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38395 } }
I20260809 22:46:22.056092 2377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:22.056226 2377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:22.056406 2377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:22.058795 2377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8180b3bb26479fb6175f9a7e2f9333" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38395 } }
I20260809 22:46:22.059561 2377 leader_election.cc:302] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f8180b3bb26479fb6175f9a7e2f9333; no voters:
I20260809 22:46:22.060252 2377 leader_election.cc:288] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:22.060366 2384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:22.061075 2384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [term 1 LEADER]: Becoming Leader. State: Replica: 7f8180b3bb26479fb6175f9a7e2f9333, State: Running, Role: LEADER
I20260809 22:46:22.061625 2384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8180b3bb26479fb6175f9a7e2f9333" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38395 } }
I20260809 22:46:22.062541 2377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:22.064466 2386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f8180b3bb26479fb6175f9a7e2f9333. Latest consensus state: current_term: 1 leader_uuid: "7f8180b3bb26479fb6175f9a7e2f9333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8180b3bb26479fb6175f9a7e2f9333" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38395 } } }
I20260809 22:46:22.064985 2386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:22.065016 2385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f8180b3bb26479fb6175f9a7e2f9333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f8180b3bb26479fb6175f9a7e2f9333" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38395 } } }
I20260809 22:46:22.065514 2385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:22.069742 2395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:22.075721 2395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:22.085398 2395 catalog_manager.cc:1347] Generated new cluster ID: 6301ab977e3d4fe4af8a09fe6fb0a600
I20260809 22:46:22.085541 2395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:22.099092 2395 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:22.101099 2395 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:22.111943 2395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f8180b3bb26479fb6175f9a7e2f9333: Generated new TSK 0
I20260809 22:46:22.113166 2395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:46:22.138227 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:22.129872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.24.136.62:38395)
0809 22:46:22.130081 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:22.130088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:22.130407 (+ 319us) server_negotiation.cc:407] Connection header received
0809 22:46:22.130647 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:22.130650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:22.130724 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:22.130879 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:46:22.131490 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.132225 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.134228 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.134683 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.134794 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:22.135448 (+ 654us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:46:22.137531 (+ 2083us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 22:46:22.137884 (+ 353us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46420: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 22:46:22.138374 2380 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46420: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 22:46:22.138444 2432 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:22.129610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38395 (local address 127.0.0.1:46420)
0809 22:46:22.130259 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:22.130304 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:22.130477 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:22.130922 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:22.130928 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:22.130942 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:46:22.131262 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:22.131289 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.132378 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.132384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:22.134075 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:22.134092 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.134885 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.134893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:22.135086 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:22.135138 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:46:22.137744 (+ 2606us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 22:46:22.138009 (+ 265us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.136.62:38395: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46825/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":350,"threads_started":1}
I20260809 22:46:22.139354 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 2253
I20260809 22:46:22.140729 2362 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 22:46:22.150698 2434 webserver.cc:533] Webserver: sslize failed:
E20260809 22:46:22.150823 2434 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:46:22.232749 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 2253
2026-08-09T22:46:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (758 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T22:46:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:22Z Disabled control of system clock
I20260809 22:46:22.334209 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:33057
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:39525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:33057
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/testchainca.cert with env {}
W20260809 22:46:22.688248 2441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:22.688570 2441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:22.688659 2441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:22.695998 2441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:22.696127 2441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:22.696164 2441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:22.696216 2441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:22.696264 2441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:46:22.696324 2441 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:46:22.696363 2441 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 22:46:22.706656 2441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:39525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:33057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:22.708161 2441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:22.709503 2441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:22.721947 2453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:22.721818 2462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:22.721845 2457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:22.722599 2441 server_base.cc:1034] running on GCE node
I20260809 22:46:22.723577 2441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:22.725185 2441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:22.726409 2441 hybrid_clock.cc:648] HybridClock initialized: now 1786315582726359 us; error 46 us; skew 500 ppm
I20260809 22:46:22.726794 2441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:46:22.729300 2441 webserver.cc:460] Webserver started at http://127.24.136.62:39183/ using document root <none> and password file <none>
I20260809 22:46:22.730037 2441 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:22.730141 2441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:22.730475 2441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:22.733583 2441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "bb81816b4827433bbd3f40e89b04ab16"
format_stamp: "Formatted at 2026-08-09 22:46:22 on dist-test-slave-pww0"
server_key: "46f30e68335747005295a027ced3f87c"
server_key_iv: "910e5e818f743c868bf2988f07b48fb7"
server_key_version: "encryptionkey@0"
I20260809 22:46:22.734445 2441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "bb81816b4827433bbd3f40e89b04ab16"
format_stamp: "Formatted at 2026-08-09 22:46:22 on dist-test-slave-pww0"
server_key: "46f30e68335747005295a027ced3f87c"
server_key_iv: "910e5e818f743c868bf2988f07b48fb7"
server_key_version: "encryptionkey@0"
I20260809 22:46:22.740257 2441 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 22:46:22.744340 2470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:22.746157 2441 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:46:22.746374 2441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "bb81816b4827433bbd3f40e89b04ab16"
format_stamp: "Formatted at 2026-08-09 22:46:22 on dist-test-slave-pww0"
server_key: "46f30e68335747005295a027ced3f87c"
server_key_iv: "910e5e818f743c868bf2988f07b48fb7"
server_key_version: "encryptionkey@0"
I20260809 22:46:22.746624 2441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:22.767998 2441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:22.768929 2441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:22.769213 2441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:22.787683 2441 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:33057
I20260809 22:46:22.787701 2548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:33057 every 8 connection(s)
I20260809 22:46:22.789178 2441 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:22.789934 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 2441
I20260809 22:46:22.790117 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:22.790941 25120 external_mini_cluster.cc:1428] Setting key 6cd92442197d6d2a78bf8a0de4f9d256
I20260809 22:46:22.794049 2549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:22.808460 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:22.793112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.24.136.62:33057)
0809 22:46:22.794257 (+ 1145us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:22.794275 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:22.794334 (+ 59us) server_negotiation.cc:407] Connection header received
0809 22:46:22.796700 (+ 2366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:22.796737 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:22.797437 (+ 700us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:22.798069 (+ 632us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:22.798765 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.801062 (+ 2297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.802616 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.803820 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.804022 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:22.805234 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:22.805246 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:22.805268 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:46:22.806101 (+ 833us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:22.806214 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:46:22.806245 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:22.806358 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:22.806577 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 22:46:22.806996 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":356,"threads_started":1}
I20260809 22:46:22.809612 2549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16: Bootstrap starting.
I20260809 22:46:22.812860 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:46:22.813485 2549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:22.814548 2549 log.cc:824] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:22.817965 2549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16: No bootstrap required, opened a new log
I20260809 22:46:22.826274 2549 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb81816b4827433bbd3f40e89b04ab16" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33057 } }
I20260809 22:46:22.826735 2549 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:22.826846 2549 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb81816b4827433bbd3f40e89b04ab16, State: Initialized, Role: FOLLOWER
I20260809 22:46:22.827510 2549 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb81816b4827433bbd3f40e89b04ab16" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33057 } }
I20260809 22:46:22.827821 2549 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:22.827987 2549 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:22.828172 2549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:22.830615 2549 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb81816b4827433bbd3f40e89b04ab16" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33057 } }
I20260809 22:46:22.831154 2549 leader_election.cc:302] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb81816b4827433bbd3f40e89b04ab16; no voters:
I20260809 22:46:22.831826 2549 leader_election.cc:288] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:22.831985 2556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:22.832662 2556 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [term 1 LEADER]: Becoming Leader. State: Replica: bb81816b4827433bbd3f40e89b04ab16, State: Running, Role: LEADER
I20260809 22:46:22.833211 2556 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb81816b4827433bbd3f40e89b04ab16" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33057 } }
I20260809 22:46:22.834007 2549 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:22.836352 2558 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb81816b4827433bbd3f40e89b04ab16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb81816b4827433bbd3f40e89b04ab16" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33057 } } }
I20260809 22:46:22.836736 2558 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:22.837580 2570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:22.837394 2561 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb81816b4827433bbd3f40e89b04ab16. Latest consensus state: current_term: 1 leader_uuid: "bb81816b4827433bbd3f40e89b04ab16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb81816b4827433bbd3f40e89b04ab16" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33057 } } }
I20260809 22:46:22.838569 2561 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:22.843400 2570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:22.854442 2570 catalog_manager.cc:1347] Generated new cluster ID: ca2e59d6692d4421aa534cc674e75f3d
I20260809 22:46:22.854600 2570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:22.873631 2570 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:22.875033 2570 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:22.885257 2570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb81816b4827433bbd3f40e89b04ab16: Generated new TSK 0
I20260809 22:46:22.886181 2570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:46:23.426904 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:22.900718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.24.136.62:33057)
0809 22:46:22.900877 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:22.900882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:22.901256 (+ 374us) server_negotiation.cc:407] Connection header received
0809 22:46:22.901522 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:22.901526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:22.901603 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:22.901731 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:46:22.902279 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.903062 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.905202 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.905691 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.905799 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:22.906361 (+ 562us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:46:23.426299 (+519938us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 22:46:23.426650 (+ 351us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44428: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 22:46:23.427017 2552 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44428: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 22:46:23.427248 2599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:22.900521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33057 (local address 127.0.0.1:44428)
0809 22:46:22.901147 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:22.901167 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:22.901342 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:22.901765 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:22.901770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:22.901784 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:46:22.902083 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:22.902095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.903208 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.903214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:22.905025 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:22.905042 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:22.905864 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:22.905870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:22.906056 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:22.906106 (+ 50us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:46:23.426626 (+520520us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 22:46:23.426883 (+ 257us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.136.62:33057: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":354,"threads_started":1}
I20260809 22:46:23.428620 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 2441
I20260809 22:46:23.429847 2544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:23.533780 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 2441
2026-08-09T22:46:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1283 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (432 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (441 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:24 dist-test-slave-pww0 krb5kdc[2624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:24 dist-test-slave-pww0 krb5kdc[2624](info): set up 2 sockets
Aug 09 22:46:24 dist-test-slave-pww0 krb5kdc[2624](info): commencing operation
krb5kdc: starting...
W20260809 22:46:25.663153 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:46:25 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:46:25.921548 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:36709
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:36709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:26.273969 2649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:26.274298 2649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:26.274446 2649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:26.282035 2649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:26.282164 2649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:26.282241 2649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:26.282323 2649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:26.282387 2649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:26.292721 2649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:36709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:36709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:26.294116 2649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:26.295549 2649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:26.308192 2661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:26.308109 2665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:26.308156 2658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:26.308894 2649 server_base.cc:1034] running on GCE node
I20260809 22:46:26.309768 2649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:26.311262 2649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:26.312467 2649 hybrid_clock.cc:648] HybridClock initialized: now 1786315586312407 us; error 49 us; skew 500 ppm
I20260809 22:46:26.312824 2649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:26 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:26.318755 2649 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:46:26.320439 2649 webserver.cc:460] Webserver started at http://127.24.136.62:35061/ using document root <none> and password file <none>
I20260809 22:46:26.321156 2649 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:26.321290 2649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:26.321609 2649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:26.324820 2649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "28b4104803e247aeb3a9ef673f6dac5f"
format_stamp: "Formatted at 2026-08-09 22:46:26 on dist-test-slave-pww0"
server_key: "cc5aea926d2f05b853f15c5fec93ab9e"
server_key_iv: "fdf3887b472b7d59424aed48496a0fa8"
server_key_version: "encryptionkey@0"
I20260809 22:46:26.325646 2649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "28b4104803e247aeb3a9ef673f6dac5f"
format_stamp: "Formatted at 2026-08-09 22:46:26 on dist-test-slave-pww0"
server_key: "cc5aea926d2f05b853f15c5fec93ab9e"
server_key_iv: "fdf3887b472b7d59424aed48496a0fa8"
server_key_version: "encryptionkey@0"
I20260809 22:46:26.331445 2649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:46:26.335633 2679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:26.337208 2649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:26.337400 2649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "28b4104803e247aeb3a9ef673f6dac5f"
format_stamp: "Formatted at 2026-08-09 22:46:26 on dist-test-slave-pww0"
server_key: "cc5aea926d2f05b853f15c5fec93ab9e"
server_key_iv: "fdf3887b472b7d59424aed48496a0fa8"
server_key_version: "encryptionkey@0"
I20260809 22:46:26.337605 2649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:26.357019 2649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:26.361863 2649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:26.362159 2649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:26.381083 2649 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:36709
I20260809 22:46:26.381121 2767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:36709 every 8 connection(s)
I20260809 22:46:26.382563 2649 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:26.386806 2768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:26.388417 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 2649
I20260809 22:46:26.388628 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:26.389293 25120 external_mini_cluster.cc:1428] Setting key e670c0b847052f9279db7675c6b981b4
I20260809 22:46:26.400115 2768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f: Bootstrap starting.
I20260809 22:46:26.404434 2768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f: Neither blocks nor log segments found. Creating new log.
Aug 09 22:46:26 dist-test-slave-pww0 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:26.405622 2768 log.cc:824] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:26.409610 2768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f: No bootstrap required, opened a new log
I20260809 22:46:26.415706 2778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:26.391427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.24.136.62:36709)
0809 22:46:26.392314 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:26.392330 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:26.392379 (+ 49us) server_negotiation.cc:407] Connection header received
0809 22:46:26.394172 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:26.394201 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:26.394642 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:26.394993 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:26.397074 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:26.398517 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:26.400355 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:26.401095 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:26.401244 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:26.406118 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:26.406133 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:26.406155 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:26.406214 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:26.410330 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:26.411194 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:26.411205 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:26.411215 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:26.411483 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:26.411964 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:26.411969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:26.411972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:26.412825 (+ 853us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:26.413146 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:26.413615 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 22:46:26.414149 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":378,"threads_started":1}
I20260809 22:46:26.419337 2768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b4104803e247aeb3a9ef673f6dac5f" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 36709 } }
I20260809 22:46:26.419796 2768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:26.419878 2768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b4104803e247aeb3a9ef673f6dac5f, State: Initialized, Role: FOLLOWER
I20260809 22:46:26.420401 2768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b4104803e247aeb3a9ef673f6dac5f" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 36709 } }
I20260809 22:46:26.420609 2768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:26.420709 2768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:26.420831 2768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:26.423039 2768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b4104803e247aeb3a9ef673f6dac5f" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 36709 } }
I20260809 22:46:26.423612 2768 leader_election.cc:302] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28b4104803e247aeb3a9ef673f6dac5f; no voters:
I20260809 22:46:26.424222 2768 leader_election.cc:288] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:26.424404 2781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:26.425357 2781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [term 1 LEADER]: Becoming Leader. State: Replica: 28b4104803e247aeb3a9ef673f6dac5f, State: Running, Role: LEADER
I20260809 22:46:26.425980 2781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b4104803e247aeb3a9ef673f6dac5f" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 36709 } }
I20260809 22:46:26.426523 2768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:26.428925 2785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b4104803e247aeb3a9ef673f6dac5f. Latest consensus state: current_term: 1 leader_uuid: "28b4104803e247aeb3a9ef673f6dac5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b4104803e247aeb3a9ef673f6dac5f" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 36709 } } }
I20260809 22:46:26.429418 2785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:26.428917 2784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28b4104803e247aeb3a9ef673f6dac5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b4104803e247aeb3a9ef673f6dac5f" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 36709 } } }
I20260809 22:46:26.431680 2784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:26.432174 2792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:26.439812 2792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:26.449745 2792 catalog_manager.cc:1347] Generated new cluster ID: 627e815b2dd348e886a92231346086c5
I20260809 22:46:26.449898 2792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:26.464308 2792 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:26.465747 2792 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:26.474795 2792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28b4104803e247aeb3a9ef673f6dac5f: Generated new TSK 0
I20260809 22:46:26.475818 2792 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:46:26.541682 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:36709
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:26.896677 2824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:26.897001 2824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:26.897176 2824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:26.904685 2824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:26.904812 2824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:26.904991 2824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:26.915602 2824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:36709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:26.917045 2824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:26.918481 2824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:26.935838 2839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:26.935751 2840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:26.935776 2844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:26.936051 2824 server_base.cc:1034] running on GCE node
I20260809 22:46:26.936892 2824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:26.937959 2824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:26.939180 2824 hybrid_clock.cc:648] HybridClock initialized: now 1786315586939111 us; error 58 us; skew 500 ppm
I20260809 22:46:26.939556 2824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:26 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:26.945945 2824 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:46:26.947736 2824 webserver.cc:460] Webserver started at http://127.24.136.1:43913/ using document root <none> and password file <none>
I20260809 22:46:26.948511 2824 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:26.948660 2824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:26.949003 2824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:26.952409 2824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "173ffdfe11224e86a7ce0b990b9cb1c7"
format_stamp: "Formatted at 2026-08-09 22:46:26 on dist-test-slave-pww0"
server_key: "cd258daa2c0f0bc3bed0d606dd618996"
server_key_iv: "727e3fa9aee02f11674938710264d163"
server_key_version: "encryptionkey@0"
I20260809 22:46:26.953305 2824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "173ffdfe11224e86a7ce0b990b9cb1c7"
format_stamp: "Formatted at 2026-08-09 22:46:26 on dist-test-slave-pww0"
server_key: "cd258daa2c0f0bc3bed0d606dd618996"
server_key_iv: "727e3fa9aee02f11674938710264d163"
server_key_version: "encryptionkey@0"
I20260809 22:46:26.959172 2824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 22:46:26.963095 2856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:26.964607 2824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:26.964818 2824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "173ffdfe11224e86a7ce0b990b9cb1c7"
format_stamp: "Formatted at 2026-08-09 22:46:26 on dist-test-slave-pww0"
server_key: "cd258daa2c0f0bc3bed0d606dd618996"
server_key_iv: "727e3fa9aee02f11674938710264d163"
server_key_version: "encryptionkey@0"
I20260809 22:46:26.965044 2824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:26.987229 2824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:26.991780 2824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:26.992062 2824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:26.993067 2824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:26.994607 2824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:26.994701 2824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:26.994817 2824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:26.994861 2824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:27.032609 2824 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:36953
I20260809 22:46:27.032671 3043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:36953 every 8 connection(s)
I20260809 22:46:27.034310 2824 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:27.044117 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 2824
I20260809 22:46:27.044348 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:27.045202 25120 external_mini_cluster.cc:1428] Setting key e70fa780062521e994fafc2cf74ba3bc
Aug 09 22:46:27 dist-test-slave-pww0 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:27.064872 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:27.039138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:60741 (local address 127.24.136.62:36709)
0809 22:46:27.039642 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:27.039649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:27.039673 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:46:27.042128 (+ 2455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:27.042135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:27.042265 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:27.042425 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:27.044654 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:27.045684 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:27.047883 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:27.048537 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:27.048705 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:27.055858 (+ 7153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:27.055867 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:27.055872 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:27.055927 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:27.060218 (+ 4291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:27.061700 (+ 1482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:27.061707 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:27.061711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:27.061922 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:27.062622 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:27.062628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:27.062631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:27.062964 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:27.063149 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:27.064119 (+ 970us) server_negotiation.cc:299] Negotiation successful
0809 22:46:27.064449 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":233,"threads_started":1}
I20260809 22:46:27.066493 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:27.038395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:36709 (local address 127.24.136.1:60741)
0809 22:46:27.039498 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:27.039553 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:27.041719 (+ 2166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:27.042674 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:27.042685 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:27.043526 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:27.044425 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:27.044457 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:27.045890 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:27.045896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:27.047660 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:27.047679 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:27.048829 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:27.048838 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:27.049155 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:27.050026 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:27.050065 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:27.054384 (+ 4319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:27.060526 (+ 6142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:27.060540 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:27.060563 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:27.061476 (+ 913us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:27.062090 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:27.062095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:27.062098 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:27.062461 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:27.063134 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:27.063145 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:27.063637 (+ 492us) client_negotiation.cc:769] Sending connection context
0809 22:46:27.063938 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 22:46:27.064610 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":426,"threads_started":1}
I20260809 22:46:27.068511 3046 heartbeater.cc:347] Connected to a master server at 127.24.136.62:36709
I20260809 22:46:27.068953 3046 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:27.070024 3046 heartbeater.cc:511] Master 127.24.136.62:36709 requested a full tablet report, sending...
I20260809 22:46:27.072960 2707 ts_manager.cc:194] Registered new tserver with Master: 173ffdfe11224e86a7ce0b990b9cb1c7 (127.24.136.1:36953)
I20260809 22:46:27.075042 2707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:60741
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:46:27.111370 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:36709
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:27.473166 3055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:27.473505 3055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:27.473639 3055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:27.481513 3055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:27.481694 3055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:27.481880 3055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:27.492466 3055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:36709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:27.493928 3055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:27.495374 3055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:27.510643 3062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:27.510604 3064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:27.510697 3061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:27.510896 3055 server_base.cc:1034] running on GCE node
I20260809 22:46:27.511873 3055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:27.512964 3055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:27.514249 3055 hybrid_clock.cc:648] HybridClock initialized: now 1786315587514182 us; error 46 us; skew 500 ppm
I20260809 22:46:27.514632 3055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:27 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:27.518112 3055 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:46:27.519876 3055 webserver.cc:460] Webserver started at http://127.24.136.2:36597/ using document root <none> and password file <none>
I20260809 22:46:27.520633 3055 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:27.520771 3055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:27.521103 3055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:27.524453 3055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "02bbfa6253fe4d25bb087efd2941f3e6"
format_stamp: "Formatted at 2026-08-09 22:46:27 on dist-test-slave-pww0"
server_key: "fc2acd10ceee652421a194b29c6b0b2d"
server_key_iv: "1c59d3d6b48899714a759b9a614f719d"
server_key_version: "encryptionkey@0"
I20260809 22:46:27.525331 3055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "02bbfa6253fe4d25bb087efd2941f3e6"
format_stamp: "Formatted at 2026-08-09 22:46:27 on dist-test-slave-pww0"
server_key: "fc2acd10ceee652421a194b29c6b0b2d"
server_key_iv: "1c59d3d6b48899714a759b9a614f719d"
server_key_version: "encryptionkey@0"
I20260809 22:46:27.531324 3055 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 22:46:27.535600 3073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:27.537134 3055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:27.537328 3055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "02bbfa6253fe4d25bb087efd2941f3e6"
format_stamp: "Formatted at 2026-08-09 22:46:27 on dist-test-slave-pww0"
server_key: "fc2acd10ceee652421a194b29c6b0b2d"
server_key_iv: "1c59d3d6b48899714a759b9a614f719d"
server_key_version: "encryptionkey@0"
I20260809 22:46:27.537578 3055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:27.558635 3055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:27.563449 3055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:27.563814 3055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:27.564854 3055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:27.566359 3055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:27.566437 3055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:27.566529 3055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:27.566572 3055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:27.600761 3055 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:43945
I20260809 22:46:27.600831 3255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:43945 every 8 connection(s)
I20260809 22:46:27.602172 3055 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:27.609205 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 3055
I20260809 22:46:27.609376 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:27.609833 25120 external_mini_cluster.cc:1428] Setting key d600e73ae4c44f0e0b8bbe98b6412107
Aug 09 22:46:27 dist-test-slave-pww0 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:27.624316 3259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:27.604893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:53269 (local address 127.24.136.62:36709)
0809 22:46:27.605370 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:27.605377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:27.606502 (+ 1125us) server_negotiation.cc:407] Connection header received
0809 22:46:27.608423 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:27.608428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:27.608504 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:27.608610 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:27.610479 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:27.611385 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:27.613112 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:27.613746 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:27.613888 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:27.617731 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:27.617739 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:27.617744 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:27.617800 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:27.620906 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:27.621824 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:27.621829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:27.621831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:27.621979 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:27.622475 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:27.622478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:27.622481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:27.622781 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:27.622899 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:27.623842 (+ 943us) server_negotiation.cc:299] Negotiation successful
0809 22:46:27.624040 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":226,"threads_started":1}
I20260809 22:46:27.625803 3261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:27.605540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:36709 (local address 127.24.136.2:53269)
0809 22:46:27.606331 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:27.606370 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:27.608133 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:27.608905 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:27.608914 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:27.609551 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:27.610258 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:27.610290 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:27.611569 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:27.611573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:27.612942 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:27.612957 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:27.613998 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:27.614003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:27.614226 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:27.614915 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:27.614946 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:27.617436 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:27.621100 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:27.621109 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:27.621125 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:27.621672 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:27.622085 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:27.622089 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:27.622093 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:27.622376 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:27.622906 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:27.622914 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:27.623406 (+ 492us) client_negotiation.cc:769] Sending connection context
0809 22:46:27.623759 (+ 353us) client_negotiation.cc:240] Negotiation successful
0809 22:46:27.624358 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":304,"threads_started":1}
I20260809 22:46:27.627593 3256 heartbeater.cc:347] Connected to a master server at 127.24.136.62:36709
I20260809 22:46:27.627950 3256 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:27.628743 3256 heartbeater.cc:511] Master 127.24.136.62:36709 requested a full tablet report, sending...
I20260809 22:46:27.630352 2707 ts_manager.cc:194] Registered new tserver with Master: 02bbfa6253fe4d25bb087efd2941f3e6 (127.24.136.2:43945)
I20260809 22:46:27.631225 2707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:53269
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:46:27.670286 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:36709
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:28.025460 3266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:28.025779 3266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:28.025909 3266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:28.035822 3266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:28.035982 3266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:28.036163 3266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:28.046976 3266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:36709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:28.048393 3266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:28.049754 3266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:28.063655 3273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:28.063627 3274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:28.063596 3277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:28.064805 3266 server_base.cc:1034] running on GCE node
I20260809 22:46:28.065511 3266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:28.066553 3266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:28.067803 3266 hybrid_clock.cc:648] HybridClock initialized: now 1786315588067740 us; error 47 us; skew 500 ppm
I20260809 22:46:28.068177 3266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:28.071637 3266 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:46:28.073244 3266 webserver.cc:460] Webserver started at http://127.24.136.3:45351/ using document root <none> and password file <none>
I20260809 22:46:28.073993 3266 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:28.074131 3266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:28.074458 3266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:28.077644 3266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "5447748935d040fe8d1a07f781115b19"
format_stamp: "Formatted at 2026-08-09 22:46:28 on dist-test-slave-pww0"
server_key: "3c471631525b0ea4be0239bf7e5d0300"
server_key_iv: "95627ffd2460b04d02680104720102db"
server_key_version: "encryptionkey@0"
I20260809 22:46:28.078493 3266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "5447748935d040fe8d1a07f781115b19"
format_stamp: "Formatted at 2026-08-09 22:46:28 on dist-test-slave-pww0"
server_key: "3c471631525b0ea4be0239bf7e5d0300"
server_key_iv: "95627ffd2460b04d02680104720102db"
server_key_version: "encryptionkey@0"
I20260809 22:46:28.079051 3046 heartbeater.cc:503] Master 127.24.136.62:36709 was elected leader, sending a full tablet report...
I20260809 22:46:28.084674 3266 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 22:46:28.088581 3288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:28.090164 3266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:28.090364 3266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "5447748935d040fe8d1a07f781115b19"
format_stamp: "Formatted at 2026-08-09 22:46:28 on dist-test-slave-pww0"
server_key: "3c471631525b0ea4be0239bf7e5d0300"
server_key_iv: "95627ffd2460b04d02680104720102db"
server_key_version: "encryptionkey@0"
I20260809 22:46:28.090612 3266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:28.112908 3266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:28.117687 3266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:28.117992 3266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:28.119087 3266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:28.120684 3266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:28.120775 3266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:28.120903 3266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:28.120956 3266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:28.153986 3266 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:40755
I20260809 22:46:28.154039 3474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:40755 every 8 connection(s)
I20260809 22:46:28.155391 3266 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:28.157485 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 3266
I20260809 22:46:28.157675 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:28.158125 25120 external_mini_cluster.cc:1428] Setting key 166d3c1b7871248e942813955477292a
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:28.178409 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:28.158529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:36117 (local address 127.24.136.62:36709)
0809 22:46:28.159122 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.159128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.160469 (+ 1341us) server_negotiation.cc:407] Connection header received
0809 22:46:28.162679 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.162684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.162765 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.162880 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.164725 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.165588 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.167250 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.167720 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.167841 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:28.172189 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:28.172196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:28.172201 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:28.172263 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:28.175335 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.176258 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.176262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.176265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.176401 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.176805 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.176808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.176810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.177077 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:28.177197 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:28.177922 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 22:46:28.178107 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":288,"threads_started":1}
I20260809 22:46:28.179384 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:28.159182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:36709 (local address 127.24.136.3:36117)
0809 22:46:28.160305 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:28.160361 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:28.162365 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:28.163168 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:28.163177 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:28.163872 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:28.164522 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:28.164542 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.165757 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.165761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:28.167094 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:28.167110 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.167980 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.167986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:28.168232 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:28.169047 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:28.169079 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:28.171886 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:28.175565 (+ 3679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:28.175574 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:28.175591 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:28.176125 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:28.176487 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:28.176490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:28.176492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:28.176703 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:28.177174 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:28.177180 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:28.177563 (+ 383us) client_negotiation.cc:769] Sending connection context
0809 22:46:28.177808 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 22:46:28.178245 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":460,"threads_started":1}
I20260809 22:46:28.180923 3477 heartbeater.cc:347] Connected to a master server at 127.24.136.62:36709
I20260809 22:46:28.181245 3477 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:28.182019 3477 heartbeater.cc:511] Master 127.24.136.62:36709 requested a full tablet report, sending...
I20260809 22:46:28.183634 2707 ts_manager.cc:194] Registered new tserver with Master: 5447748935d040fe8d1a07f781115b19 (127.24.136.3:40755)
I20260809 22:46:28.184480 2707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:36117
I20260809 22:46:28.186890 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/test-admin.keytab.
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:28.217697 25120 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 22:46:28.223246 3490 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:28.220488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.24.136.1:36953)
0809 22:46:28.221174 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.221184 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.221219 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:46:28.221318 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.221325 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.221550 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.221698 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.222828 (+ 1130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54246: BlockingRecv error: recv got EOF from 127.0.0.1:54246 (error 108)
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":313,"threads_started":1}
W20260809 22:46:28.223486 25120 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5cc' of type FILE
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:28.225741 25120 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:28.227283 25120 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:46:28.243943 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:28.229063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54262 (local address 127.24.136.1:36953)
0809 22:46:28.229219 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.229224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.229636 (+ 412us) server_negotiation.cc:407] Connection header received
0809 22:46:28.229801 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.229805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.229873 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.229963 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.231066 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.232164 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.233474 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.234089 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.234195 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:28.238067 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:28.238084 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:28.238102 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:28.238145 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:28.241138 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.241945 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.241956 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.241959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.242104 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.242456 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.242459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.242461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.242850 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:28.243089 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:28.243369 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 22:46:28.243654 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 22:46:28.250145 3490 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:28.248235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.24.136.1:36953)
0809 22:46:28.248410 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.248416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.248771 (+ 355us) server_negotiation.cc:407] Connection header received
0809 22:46:28.248950 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.248954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.249020 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.249101 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.249906 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54266: BlockingRecv error: recv got EOF from 127.0.0.1:54266 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 22:46:28.250309 25120 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5cc' of type FILE
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:28.251947 25120 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:28.253331 25120 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:46:28.268432 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:28.254902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.24.136.1:36953)
0809 22:46:28.255141 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.255147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.255316 (+ 169us) server_negotiation.cc:407] Connection header received
0809 22:46:28.255530 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.255535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.255608 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.255700 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.256709 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.257714 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.259060 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.259617 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.259730 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:28.263528 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:28.263533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:28.263537 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:28.263579 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:28.265973 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.266698 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.266702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.266704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.266860 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.267183 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.267186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.267187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.267521 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:28.267725 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:28.267995 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 22:46:28.268196 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 22:46:28.273777 3490 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:28.271821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54282 (local address 127.24.136.1:36953)
0809 22:46:28.272011 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.272017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.272341 (+ 324us) server_negotiation.cc:407] Connection header received
0809 22:46:28.272516 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.272521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.272610 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.272695 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.273521 (+ 826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54282: BlockingRecv error: recv got EOF from 127.0.0.1:54282 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 22:46:28.274266 25120 init.cc:244] Error while opening credential cache '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315533505708-25120-0/krb5kdc/krb5cc)
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:28.275960 25120 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:28.277339 25120 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:46:28.292573 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:28.279042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.24.136.1:36953)
0809 22:46:28.279292 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.279298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.279441 (+ 143us) server_negotiation.cc:407] Connection header received
0809 22:46:28.279662 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.279666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.279741 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.279822 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.280762 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.281503 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.282714 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.283137 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.283234 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:28.286833 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:28.286838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:28.286842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:28.286883 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:28.289569 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.290311 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.290316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.290318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.290488 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.290801 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.290804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.290806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.291136 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:28.291390 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:28.291991 (+ 601us) server_negotiation.cc:299] Negotiation successful
0809 22:46:28.292236 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 22:46:28.294333 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 2824
I20260809 22:46:28.307099 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:28.295102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54296 (local address 127.24.136.1:36953)
0809 22:46:28.295313 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.295320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.295673 (+ 353us) server_negotiation.cc:407] Connection header received
0809 22:46:28.295872 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.295875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.295938 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.296014 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.297061 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.297764 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.299073 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.299568 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.299682 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:28.301835 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:28.301840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:28.301844 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:28.301882 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:28.304585 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.305310 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.305314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.305317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.305460 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.305761 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.305764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.305766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.306064 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:28.306381 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:28.306575 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 22:46:28.306758 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:46:28.307799 3038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:28.391572 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 2824
I20260809 22:46:28.418661 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 3055
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:46:28.435698 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:28.419699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.24.136.2:43945)
0809 22:46:28.420274 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.420281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.420307 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:46:28.420373 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.420379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.420581 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.420719 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.421860 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.422891 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.424332 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.424933 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.425051 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:28.429111 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:28.429127 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:28.429145 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:28.429194 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:28.432298 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.433083 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.433091 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.433093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.433243 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.433829 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.433836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.433838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.434269 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:28.434535 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:28.435066 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 22:46:28.435326 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":272,"threads_started":1}
I20260809 22:46:28.437083 3246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:28.521318 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 3055
I20260809 22:46:28.548053 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 3266
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[2624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:46:28.564610 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:28.549086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.24.136.3:40755)
0809 22:46:28.549717 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:28.549726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:28.549759 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:46:28.549821 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:28.549827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:28.550041 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:28.550193 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:28.551313 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.552432 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.553819 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:28.554430 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:28.554536 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:28.558652 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:28.558669 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:28.558686 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:28.558738 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:28.561788 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.562600 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.562607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.562610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.562750 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:28.563081 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:28.563083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:28.563085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:28.563505 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:28.563749 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:28.563993 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 22:46:28.564264 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":323,"threads_started":1}
I20260809 22:46:28.565920 3467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:28.652915 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 3266
I20260809 22:46:28.678990 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 2649
I20260809 22:46:28.680269 2761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:28.785583 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 2649
2026-08-09T22:46:28Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4375 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[3516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[3516](info): set up 2 sockets
Aug 09 22:46:28 dist-test-slave-pww0 krb5kdc[3516](info): commencing operation
krb5kdc: starting...
W20260809 22:46:30.041698 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:46:30 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.62.keytab.
Entry for principal oryx/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.62.keytab.
I20260809 22:46:30.307852 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:42691
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.62.keytab
--principal=oryx/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:42691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:30.662457 3538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:30.662824 3538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:30.662972 3538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:30.670544 3538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:30.670701 3538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:30.670791 3538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:30.670871 3538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:30.670934 3538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:30.681298 3538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:42691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.62.keytab
--principal=oryx/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:42691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:30.682787 3538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:30.684207 3538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:30.697629 3548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:30.697592 3549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:30.697985 3552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:30.698763 3538 server_base.cc:1034] running on GCE node
I20260809 22:46:30.699705 3538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:30.701416 3538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:30.702674 3538 hybrid_clock.cc:648] HybridClock initialized: now 1786315590702588 us; error 73 us; skew 500 ppm
I20260809 22:46:30.703152 3538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:30 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:30.706650 3538 init.cc:377] Logged in from keytab as oryx/127.24.136.62@KRBTEST.COM (short username oryx)
I20260809 22:46:30.708475 3538 webserver.cc:460] Webserver started at http://127.24.136.62:34197/ using document root <none> and password file <none>
I20260809 22:46:30.709227 3538 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:30.709362 3538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:30.709687 3538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:30.713105 3538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d"
format_stamp: "Formatted at 2026-08-09 22:46:30 on dist-test-slave-pww0"
server_key: "9eda5aad25a7660f8a033e5ce38e88d8"
server_key_iv: "be6be305c74915da4f0234e20f2a07f1"
server_key_version: "encryptionkey@0"
I20260809 22:46:30.714058 3538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d"
format_stamp: "Formatted at 2026-08-09 22:46:30 on dist-test-slave-pww0"
server_key: "9eda5aad25a7660f8a033e5ce38e88d8"
server_key_iv: "be6be305c74915da4f0234e20f2a07f1"
server_key_version: "encryptionkey@0"
I20260809 22:46:30.720605 3538 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 22:46:30.724939 3565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:30.726719 3538 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:46:30.726959 3538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d"
format_stamp: "Formatted at 2026-08-09 22:46:30 on dist-test-slave-pww0"
server_key: "9eda5aad25a7660f8a033e5ce38e88d8"
server_key_iv: "be6be305c74915da4f0234e20f2a07f1"
server_key_version: "encryptionkey@0"
I20260809 22:46:30.727185 3538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:30.755447 3538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:30.760267 3538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:30.760568 3538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:30.780010 3538 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:42691
I20260809 22:46:30.780041 3655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:42691 every 8 connection(s)
I20260809 22:46:30.781534 3538 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:30.785072 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 3538
I20260809 22:46:30.785311 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:30.785735 3656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:30.785998 25120 external_mini_cluster.cc:1428] Setting key b4f070870f8d4c25a0291476c9a4a2f2
I20260809 22:46:30.801620 3656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Bootstrap starting.
Aug 09 22:46:30 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:30.805881 3656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:30.807421 3656 log.cc:824] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:30.811299 3656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: No bootstrap required, opened a new log
I20260809 22:46:30.812592 3660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:30.788177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48966 (local address 127.24.136.62:42691)
0809 22:46:30.789116 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:30.789133 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:30.789194 (+ 61us) server_negotiation.cc:407] Connection header received
0809 22:46:30.791056 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:30.791092 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:30.791758 (+ 666us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:30.792286 (+ 528us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:30.793756 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:30.795825 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:30.797258 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:30.798435 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:30.798658 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:30.803173 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:30.803188 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:30.803215 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:30.803291 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:30.807424 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:30.808476 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:30.808484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:30.808491 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:30.808686 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:30.809146 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:30.809151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:30.809153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:30.809902 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:30.810231 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:30.810655 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 22:46:30.811139 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":435,"threads_started":1}
I20260809 22:46:30.820024 3656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } }
I20260809 22:46:30.820492 3656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:30.820602 3656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b23e1a1756f4ebabb8c7d61ce3b1f9d, State: Initialized, Role: FOLLOWER
I20260809 22:46:30.821201 3656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } }
I20260809 22:46:30.821463 3656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:30.821593 3656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:30.821753 3656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:30.824123 3656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } }
I20260809 22:46:30.824698 3656 leader_election.cc:302] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b23e1a1756f4ebabb8c7d61ce3b1f9d; no voters:
I20260809 22:46:30.825356 3656 leader_election.cc:288] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:30.825538 3663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:30.826650 3663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 1 LEADER]: Becoming Leader. State: Replica: 8b23e1a1756f4ebabb8c7d61ce3b1f9d, State: Running, Role: LEADER
I20260809 22:46:30.827205 3663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } }
I20260809 22:46:30.827703 3656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:30.830404 3664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } }
I20260809 22:46:30.830379 3667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b23e1a1756f4ebabb8c7d61ce3b1f9d. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } }
I20260809 22:46:30.830855 3664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:30.830853 3667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:30.836102 3672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:30.841224 3672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:30.851284 3672 catalog_manager.cc:1347] Generated new cluster ID: dd6b1a6cdddb4375a36414e5c98b8041
I20260809 22:46:30.851428 3672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:30.864140 3672 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:30.866207 3672 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:30.885821 3672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Generated new TSK 0
I20260809 22:46:30.887058 3672 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab.
Entry for principal oryx/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab.
I20260809 22:46:30.948654 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42691
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab
--principal=oryx/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:31.308902 3692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:31.309239 3692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:31.309365 3692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:31.317212 3692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:31.317337 3692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:31.317492 3692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:31.327728 3692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab
--principal=oryx/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:31.329181 3692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:31.330686 3692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:31.344416 3701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:31.344372 3704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:31.344449 3700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:31.345420 3692 server_base.cc:1034] running on GCE node
I20260809 22:46:31.346050 3692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:31.347023 3692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:31.348213 3692 hybrid_clock.cc:648] HybridClock initialized: now 1786315591348151 us; error 59 us; skew 500 ppm
I20260809 22:46:31.348547 3692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:31 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315591, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:31.355415 3692 init.cc:377] Logged in from keytab as oryx/127.24.136.1@KRBTEST.COM (short username oryx)
I20260809 22:46:31.357125 3692 webserver.cc:460] Webserver started at http://127.24.136.1:45857/ using document root <none> and password file <none>
I20260809 22:46:31.357859 3692 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:31.357988 3692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:31.358294 3692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:31.361440 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "1b0df5f2b5834104a810228d4ccd1052"
format_stamp: "Formatted at 2026-08-09 22:46:31 on dist-test-slave-pww0"
server_key: "b8c4af1363ef8594fcbe359f43ed99c4"
server_key_iv: "16d8eb14186720dd11a642c50aea96a8"
server_key_version: "encryptionkey@0"
I20260809 22:46:31.362262 3692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "1b0df5f2b5834104a810228d4ccd1052"
format_stamp: "Formatted at 2026-08-09 22:46:31 on dist-test-slave-pww0"
server_key: "b8c4af1363ef8594fcbe359f43ed99c4"
server_key_iv: "16d8eb14186720dd11a642c50aea96a8"
server_key_version: "encryptionkey@0"
I20260809 22:46:31.368786 3692 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 22:46:31.372638 3717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:31.374135 3692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:46:31.374327 3692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "1b0df5f2b5834104a810228d4ccd1052"
format_stamp: "Formatted at 2026-08-09 22:46:31 on dist-test-slave-pww0"
server_key: "b8c4af1363ef8594fcbe359f43ed99c4"
server_key_iv: "16d8eb14186720dd11a642c50aea96a8"
server_key_version: "encryptionkey@0"
I20260809 22:46:31.374539 3692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:31.410434 3692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:31.415383 3692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:31.415747 3692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:31.416874 3692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:31.418345 3692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:31.418427 3692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:31.418557 3692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:31.418620 3692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:31.451602 3692 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:38659
I20260809 22:46:31.451700 3898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:38659 every 8 connection(s)
I20260809 22:46:31.453096 3692 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:31.456828 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 3692
I20260809 22:46:31.457057 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:31.457865 25120 external_mini_cluster.cc:1428] Setting key 92ee853949c5afbed6941fb569c7b3ee
Aug 09 22:46:31 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315591, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:31.474834 3902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:31.455736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:43161 (local address 127.24.136.62:42691)
0809 22:46:31.456198 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:31.456202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:31.457411 (+ 1209us) server_negotiation.cc:407] Connection header received
0809 22:46:31.459599 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:31.459603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:31.459678 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:31.459803 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:31.461468 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:31.462206 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:31.463938 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:31.464364 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:31.464492 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:31.468308 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:31.468314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:31.468317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:31.468356 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:31.470943 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:31.471832 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:31.471837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:31.471840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:31.471975 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:31.472553 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:31.472559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:31.472563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:31.472912 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:31.473070 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:31.474154 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0809 22:46:31.474448 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":231,"threads_started":1}
I20260809 22:46:31.475919 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:31.456382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:43161)
0809 22:46:31.457241 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:31.457283 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:31.459263 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:31.460119 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:31.460129 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:31.460674 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:31.461278 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:31.461298 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:31.462378 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:31.462382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:31.463750 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:31.463765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:31.464643 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:31.464652 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:31.464902 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:31.465635 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:31.465667 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:31.468019 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:31.471128 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:31.471140 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:31.471158 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:31.471689 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:31.472092 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:31.472097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:31.472101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:31.472314 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:31.473060 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:31.473069 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:31.473562 (+ 493us) client_negotiation.cc:769] Sending connection context
0809 22:46:31.473916 (+ 354us) client_negotiation.cc:240] Negotiation successful
0809 22:46:31.474529 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":333,"threads_started":1}
I20260809 22:46:31.477738 3900 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42691
I20260809 22:46:31.478068 3900 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:31.478996 3900 heartbeater.cc:511] Master 127.24.136.62:42691 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.24.136.2@KRBTEST.COM; defaulting to no policy
I20260809 22:46:31.481420 3603 ts_manager.cc:194] Registered new tserver with Master: 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1:38659)
I20260809 22:46:31.482976 3603 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:43161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab.
Entry for principal oryx/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab.
I20260809 22:46:31.514751 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42691
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab
--principal=oryx/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:31.865283 3908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:31.865605 3908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:31.865703 3908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:31.873097 3908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:31.873214 3908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:31.873330 3908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:31.883217 3908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab
--principal=oryx/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:31.884588 3908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:31.885890 3908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:31.899227 3921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:31.899295 3918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:31.899257 3917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:31.900220 3908 server_base.cc:1034] running on GCE node
I20260809 22:46:31.900900 3908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:31.901923 3908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:31.903129 3908 hybrid_clock.cc:648] HybridClock initialized: now 1786315591903075 us; error 51 us; skew 500 ppm
I20260809 22:46:31.903487 3908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:31 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315591, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:31.906976 3908 init.cc:377] Logged in from keytab as oryx/127.24.136.2@KRBTEST.COM (short username oryx)
I20260809 22:46:31.908615 3908 webserver.cc:460] Webserver started at http://127.24.136.2:37281/ using document root <none> and password file <none>
I20260809 22:46:31.909335 3908 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:31.909446 3908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:31.909761 3908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:31.912935 3908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "88bc57d6506741ff97979bf46d4b9959"
format_stamp: "Formatted at 2026-08-09 22:46:31 on dist-test-slave-pww0"
server_key: "b218ba34ca1188ef6ad2e97df8f14f21"
server_key_iv: "2c6aa4d48e0c94fb66f6b00b5db1058c"
server_key_version: "encryptionkey@0"
I20260809 22:46:31.913784 3908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "88bc57d6506741ff97979bf46d4b9959"
format_stamp: "Formatted at 2026-08-09 22:46:31 on dist-test-slave-pww0"
server_key: "b218ba34ca1188ef6ad2e97df8f14f21"
server_key_iv: "2c6aa4d48e0c94fb66f6b00b5db1058c"
server_key_version: "encryptionkey@0"
I20260809 22:46:31.919624 3908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 22:46:31.923393 3936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:31.924860 3908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:31.925026 3908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "88bc57d6506741ff97979bf46d4b9959"
format_stamp: "Formatted at 2026-08-09 22:46:31 on dist-test-slave-pww0"
server_key: "b218ba34ca1188ef6ad2e97df8f14f21"
server_key_iv: "2c6aa4d48e0c94fb66f6b00b5db1058c"
server_key_version: "encryptionkey@0"
I20260809 22:46:31.925236 3908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:31.942816 3908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:31.947306 3908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:31.947628 3908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:31.948695 3908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:31.950304 3908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:31.950382 3908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:31.950497 3908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:31.950547 3908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:31.982748 3908 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:45875
I20260809 22:46:31.982787 4127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:45875 every 8 connection(s)
I20260809 22:46:31.984078 3908 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:31.991776 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 3908
I20260809 22:46:31.991940 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:31.992391 25120 external_mini_cluster.cc:1428] Setting key 9832901ee03ba2c540f8c357d2db650b
Aug 09 22:46:31 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315591, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:32.005682 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:31.986649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:56445 (local address 127.24.136.62:42691)
0809 22:46:31.987115 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:31.987119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:31.987965 (+ 846us) server_negotiation.cc:407] Connection header received
0809 22:46:31.989787 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:31.989793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:31.989878 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:31.989978 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:31.991676 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:31.992511 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:31.994376 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:31.994878 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:31.994976 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:31.999366 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:31.999372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:31.999376 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:31.999412 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:32.002045 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:32.003021 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:32.003025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:32.003028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:32.003175 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:32.003720 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:32.003725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:32.003727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:32.004025 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:32.004156 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:32.005038 (+ 882us) server_negotiation.cc:299] Negotiation successful
0809 22:46:32.005296 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":249,"threads_started":1}
I20260809 22:46:32.006522 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:31.987031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.2:56445)
0809 22:46:31.987800 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:31.987841 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:31.989493 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:31.990236 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:31.990254 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:31.990794 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:31.991476 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:31.991509 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:31.992680 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:31.992686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:31.994211 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:31.994227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:31.995123 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:31.995132 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:31.995510 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:31.996255 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:31.996289 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:31.999073 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:32.002222 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:32.002232 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:32.002260 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:32.002865 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:32.003290 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:32.003295 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:32.003297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:32.003597 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:32.004150 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:32.004158 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:32.004650 (+ 492us) client_negotiation.cc:769] Sending connection context
0809 22:46:32.004902 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 22:46:32.005353 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":274,"threads_started":1}
I20260809 22:46:32.008293 4128 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42691
I20260809 22:46:32.008605 4128 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:32.009382 4128 heartbeater.cc:511] Master 127.24.136.62:42691 requested a full tablet report, sending...
I20260809 22:46:32.010897 3603 ts_manager.cc:194] Registered new tserver with Master: 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875)
I20260809 22:46:32.011785 3603 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:56445
WARNING: no policy specified for oryx/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab.
Entry for principal oryx/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab.
I20260809 22:46:32.051132 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42691
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab
--principal=oryx/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:32.396729 4137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:32.397030 4137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:32.397148 4137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:32.404531 4137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:32.404677 4137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:32.404855 4137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:32.414752 4137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab
--principal=oryx/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:32.416154 4137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:32.417546 4137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:32.432256 4145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:32.432228 4144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:32.432294 4147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:32.432924 4137 server_base.cc:1034] running on GCE node
I20260809 22:46:32.433615 4137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:32.434579 4137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:32.435767 4137 hybrid_clock.cc:648] HybridClock initialized: now 1786315592435710 us; error 56 us; skew 500 ppm
I20260809 22:46:32.436080 4137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:32 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315592, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:32.439338 4137 init.cc:377] Logged in from keytab as oryx/127.24.136.3@KRBTEST.COM (short username oryx)
I20260809 22:46:32.440910 4137 webserver.cc:460] Webserver started at http://127.24.136.3:46255/ using document root <none> and password file <none>
I20260809 22:46:32.441635 4137 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:32.441761 4137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:32.442068 4137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:32.445205 4137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "71a825ab55274756b03f4c319c77286a"
format_stamp: "Formatted at 2026-08-09 22:46:32 on dist-test-slave-pww0"
server_key: "de83d754d2663cfe702254d4637f4f4c"
server_key_iv: "3fc377d74e7d97ef5252ba1b00c25fcb"
server_key_version: "encryptionkey@0"
I20260809 22:46:32.446007 4137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "71a825ab55274756b03f4c319c77286a"
format_stamp: "Formatted at 2026-08-09 22:46:32 on dist-test-slave-pww0"
server_key: "de83d754d2663cfe702254d4637f4f4c"
server_key_iv: "3fc377d74e7d97ef5252ba1b00c25fcb"
server_key_version: "encryptionkey@0"
I20260809 22:46:32.451508 4137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:46:32.455246 4157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:32.456749 4137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:32.456907 4137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "71a825ab55274756b03f4c319c77286a"
format_stamp: "Formatted at 2026-08-09 22:46:32 on dist-test-slave-pww0"
server_key: "de83d754d2663cfe702254d4637f4f4c"
server_key_iv: "3fc377d74e7d97ef5252ba1b00c25fcb"
server_key_version: "encryptionkey@0"
I20260809 22:46:32.457094 4137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:32.474941 4137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:32.479703 4137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:32.479990 4137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:32.481034 4137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:32.482537 4137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:32.482615 4137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:32.482746 4137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:32.482802 4137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:32.486255 3900 heartbeater.cc:503] Master 127.24.136.62:42691 was elected leader, sending a full tablet report...
I20260809 22:46:32.515892 4137 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:44141
I20260809 22:46:32.515939 4343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:44141 every 8 connection(s)
I20260809 22:46:32.517350 4137 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:32.527434 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 4137
I20260809 22:46:32.527734 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:32.528318 25120 external_mini_cluster.cc:1428] Setting key f4a9fd7ef84c16d45a087efe49556566
Aug 09 22:46:32 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315592, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:32.538552 4347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:32.520088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:59347 (local address 127.24.136.62:42691)
0809 22:46:32.520549 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:32.520554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:32.521545 (+ 991us) server_negotiation.cc:407] Connection header received
0809 22:46:32.523386 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:32.523392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:32.523511 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:32.523606 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:32.525201 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:32.525993 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:32.527641 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:32.528126 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:32.528275 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:32.532384 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:32.532390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:32.532393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:32.532436 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:32.535194 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:32.536076 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:32.536082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:32.536086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:32.536219 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:32.536670 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:32.536675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:32.536678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:32.536994 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:32.537136 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:32.537931 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 22:46:32.538175 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":229,"threads_started":1}
I20260809 22:46:32.539588 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:32.520668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.3:59347)
0809 22:46:32.521387 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:32.521427 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:32.523076 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:32.523886 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:32.523895 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:32.524414 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:32.525015 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:32.525033 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:32.526146 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:32.526150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:32.527477 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:32.527497 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:32.528376 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:32.528384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:32.528667 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:32.529455 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:32.529487 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:32.532159 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:32.535383 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:32.535394 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:32.535416 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:32.535946 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:32.536333 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:32.536339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:32.536343 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:32.536544 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:32.537130 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:32.537139 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:32.537524 (+ 385us) client_negotiation.cc:769] Sending connection context
0809 22:46:32.537788 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 22:46:32.538379 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":263,"threads_started":1}
I20260809 22:46:32.541209 4345 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42691
I20260809 22:46:32.541536 4345 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:32.542392 4345 heartbeater.cc:511] Master 127.24.136.62:42691 requested a full tablet report, sending...
I20260809 22:46:32.544147 3603 ts_manager.cc:194] Registered new tserver with Master: 71a825ab55274756b03f4c319c77286a (127.24.136.3:44141)
I20260809 22:46:32.545022 3603 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:59347
I20260809 22:46:32.557799 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.136.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.61.keytab.
Entry for principal oryx/127.24.136.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.61.keytab.
I20260809 22:46:32.613345 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.61:37453
--webserver_interface=127.24.136.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.61.keytab
--principal=oryx/127.24.136.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:42691,127.24.136.61:37453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:32.963662 4355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:32.963979 4355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:32.964092 4355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:32.972766 4355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:32.972909 4355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:32.972970 4355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:32.973048 4355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:32.973096 4355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:32.984113 4355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:42691,127.24.136.61:37453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.61.keytab
--principal=oryx/127.24.136.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.61:37453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.136.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
W20260809 22:46:32.984544 4355 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.136.62:42691,127.24.136.61:37453'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 22:46:32.985644 4355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:32.987071 4355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:32.999661 4368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:32.999724 4366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:33.000077 4365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:33.000422 4355 server_base.cc:1034] running on GCE node
I20260809 22:46:33.001358 4355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:33.003003 4355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:33.004269 4355 hybrid_clock.cc:648] HybridClock initialized: now 1786315593004174 us; error 74 us; skew 500 ppm
I20260809 22:46:33.004649 4355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315593, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:33.008175 4355 init.cc:377] Logged in from keytab as oryx/127.24.136.61@KRBTEST.COM (short username oryx)
I20260809 22:46:33.009793 4355 webserver.cc:460] Webserver started at http://127.24.136.61:33679/ using document root <none> and password file <none>
I20260809 22:46:33.010576 4355 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:33.010690 4355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:33.011039 4355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:33.014303 4355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data/instance:
uuid: "1e63d468dfbe4118851b84cb8dbefe4a"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "0cd6543b064707ae3a67c058bee8adfd"
server_key_iv: "07288bbb671f8c24ece2e688e24321cc"
server_key_version: "encryptionkey@0"
I20260809 22:46:33.014536 4128 heartbeater.cc:503] Master 127.24.136.62:42691 was elected leader, sending a full tablet report...
I20260809 22:46:33.015235 4355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal/instance:
uuid: "1e63d468dfbe4118851b84cb8dbefe4a"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "0cd6543b064707ae3a67c058bee8adfd"
server_key_iv: "07288bbb671f8c24ece2e688e24321cc"
server_key_version: "encryptionkey@0"
I20260809 22:46:33.021541 4355 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 22:46:33.025422 4381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:33.026942 4355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:46:33.027130 4355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal
uuid: "1e63d468dfbe4118851b84cb8dbefe4a"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "0cd6543b064707ae3a67c058bee8adfd"
server_key_iv: "07288bbb671f8c24ece2e688e24321cc"
server_key_version: "encryptionkey@0"
I20260809 22:46:33.027354 4355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:33.057385 4355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:33.061904 4355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:33.062166 4355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:33.080577 4355 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.61:37453
I20260809 22:46:33.080598 4465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.61:37453 every 8 connection(s)
I20260809 22:46:33.082023 4355 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
I20260809 22:46:33.085960 4466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:33.091380 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 4355
I20260809 22:46:33.093130 4466 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315593, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.61@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.61@KRBTEST.COM
I20260809 22:46:33.113271 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.095868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44098 (local address 127.24.136.62:42691)
0809 22:46:33.096349 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.096356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.096940 (+ 584us) server_negotiation.cc:407] Connection header received
0809 22:46:33.097693 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.097698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.097778 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.097879 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.099279 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.100113 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.101871 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.102410 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.102502 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.106769 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.106777 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.106783 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.106831 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.109607 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.110537 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.110542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.110544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.110678 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.111151 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.111157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.111160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.111663 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.111799 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.112652 (+ 853us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.112897 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":233,"threads_started":1}
I20260809 22:46:33.114593 4469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.093971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.24.136.61:37453)
0809 22:46:33.094796 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.094809 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.094853 (+ 44us) server_negotiation.cc:407] Connection header received
0809 22:46:33.097416 (+ 2563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.097445 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.097755 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.097928 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.099400 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.100849 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.102211 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.102871 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.102985 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.107677 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.107688 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.107707 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.107757 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.110774 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.111557 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.111564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.111566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.111686 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.112060 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.112064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.112067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.112566 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.112856 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.113167 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.113387 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"spinlock_wait_cycles":23296,"thread_start_us":363,"threads_started":1}
I20260809 22:46:33.114629 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.095874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44098)
0809 22:46:33.096771 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.096828 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.097422 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.097962 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.097968 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.098730 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.099098 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.099121 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.100268 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.100278 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.101710 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.101721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.102633 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.102640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.102879 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.103646 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.103682 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.106531 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.109747 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.109757 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.109770 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.110396 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.110787 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.110791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.110795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.111009 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.111787 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.111796 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.112206 (+ 410us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.112576 (+ 370us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.113086 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":462,"threads_started":1}
I20260809 22:46:33.116741 4466 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315593, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.61@KRBTEST.COM for oryx/127.24.136.61@KRBTEST.COM
I20260809 22:46:33.130599 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.117410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.0.0.1:46084)
0809 22:46:33.117588 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.117606 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.117704 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.117986 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.117991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.118353 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.118484 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.118493 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.119500 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.119506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.120373 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.120385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.121133 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.121137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.121297 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.121857 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.121873 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.123971 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.127922 (+ 3951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.127928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.127933 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.128372 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.128790 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.128795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.128798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.128968 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.129543 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.129547 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.129696 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.129818 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.130132 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 22:46:33.130646 4469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.117609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.24.136.61:37453)
0809 22:46:33.117766 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.117770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.117786 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:46:33.117826 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.117830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.117882 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.117952 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.118610 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.119318 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.120524 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.120999 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.121116 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.124200 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.124205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.124209 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.124246 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.127743 (+ 3497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.128518 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.128522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.128524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.128667 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.129070 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.129075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.129079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.129426 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.129559 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.129949 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.130312 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":51}
I20260809 22:46:33.136044 4466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Bootstrap starting.
I20260809 22:46:33.139523 4466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:33.140621 4466 log.cc:824] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:33.143504 4466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: No bootstrap required, opened a new log
I20260809 22:46:33.150743 4466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } }
I20260809 22:46:33.151136 4466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:33.151244 4466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63d468dfbe4118851b84cb8dbefe4a, State: Initialized, Role: FOLLOWER
I20260809 22:46:33.151895 4466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } }
I20260809 22:46:33.152757 4476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } } }
I20260809 22:46:33.153062 4476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:33.153816 4466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:46:33.163264 4492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:46:33.163431 4492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:46:33.166046 4355 master_runner.cc:428] Detected that this master 1e63d468dfbe4118851b84cb8dbefe4a is joining an existing cluster
I20260809 22:46:33.166179 4355 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.136.61:37453
I20260809 22:46:33.189962 4509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.170801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44114)
0809 22:46:33.171539 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.171564 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.171770 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.172322 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.172326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.175241 (+ 2915us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.175772 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.175792 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.177470 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.177476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.179414 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.179431 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.180901 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.180908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.181093 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.181821 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.181847 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.183131 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.186821 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.186826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.186829 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.187302 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.187793 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.187797 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.187801 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.187997 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.188720 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.188724 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.188941 (+ 217us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.189130 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.189468 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":373,"threads_started":1}
I20260809 22:46:33.191128 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.170500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.24.136.62:42691)
0809 22:46:33.170721 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.170728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.171652 (+ 924us) server_negotiation.cc:407] Connection header received
0809 22:46:33.171916 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.171922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.172018 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.172086 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.176012 (+ 3926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.177238 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.179641 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.180380 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.180485 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.183356 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.183364 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.183368 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.183414 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.186654 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.187447 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.187452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.187488 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.187648 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.188151 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.188156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.188160 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.188509 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.188652 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.190507 (+ 1855us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.190775 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:46:33.200519 4469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.175027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.24.136.61:37453)
0809 22:46:33.175200 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.175207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.183260 (+ 8053us) server_negotiation.cc:407] Connection header received
0809 22:46:33.183525 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.183530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.183600 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.183663 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.185340 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.186434 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.188456 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.189192 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.189303 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.192721 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.192728 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.192733 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.192772 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.197205 (+ 4433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.198072 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.198078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.198082 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.198252 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.198761 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.198766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.198769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.199195 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.199339 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.199910 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.200142 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:46:33.201589 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.174879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.0.0.1:46092)
0809 22:46:33.183139 (+ 8260us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.183165 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.183357 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.183851 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.183856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.184827 (+ 971us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.185159 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.185173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.186652 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.186657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.188262 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.188278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.189617 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.189624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.189797 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.191441 (+ 1644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.191492 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.192533 (+ 1041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.197382 (+ 4849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.197387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.197391 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.197952 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.198399 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.198404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.198408 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.198646 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.199404 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.199409 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.199607 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.200817 (+ 1210us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.201127 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8092,"thread_start_us":7830,"threads_started":1}
WARNING: no policy specified for oryx/127.24.136.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.60@KRBTEST.COM" created.
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.62@KRBTEST.COM for oryx/127.24.136.61@KRBTEST.COM
I20260809 22:46:33.223765 4513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.197973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.0.0.1:46100)
0809 22:46:33.205107 (+ 7134us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.205167 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.205748 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.206386 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.206396 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.207355 (+ 959us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.207725 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.207742 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.209126 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.209132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.210791 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.210810 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.211804 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.211812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.212057 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.212871 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.212900 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.216020 (+ 3120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.220459 (+ 4439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.220479 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.220488 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.221148 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.221608 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.221612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.221614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.221762 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.222364 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.222374 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.222670 (+ 296us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.222914 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.223279 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6833,"thread_start_us":6665,"threads_started":1}
I20260809 22:46:33.223783 4469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.198692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.24.136.61:37453)
0809 22:46:33.200774 (+ 2082us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.200779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.205279 (+ 4500us) server_negotiation.cc:407] Connection header received
0809 22:46:33.206027 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.206034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.206187 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.206295 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.207922 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.208981 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.210946 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.211601 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.211730 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.216349 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.216358 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.216363 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.216413 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.220269 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.221304 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.221308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.221311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.221467 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.221890 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.221894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.221896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.222217 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.222368 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.223090 (+ 722us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.223353 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1973,"thread_start_us":7242,"threads_started":1}
I20260809 22:46:33.226601 3603 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.136.61:37453: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a"
member_type: NON_VOTER
last_known_addr {
host: "127.24.136.61"
port: 37453
}
attrs {
promote: true
}
}
dest_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d"
cas_config_opid_index: -1
I20260809 22:46:33.227875 3603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } }
I20260809 22:46:33.239209 4519 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1e63d468dfbe4118851b84cb8dbefe4a (127.24.136.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } } }
I20260809 22:46:33.240178 4428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:33.244786 4519 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.136.61:37453
I20260809 22:46:33.244992 4428 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Refusing update from remote peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 22:46:33.246126 4355 master.cc:537] Master@127.24.136.61:37453 shutting down...
I20260809 22:46:33.246320 4519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 22:46:33.247767 4518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } } }
I20260809 22:46:33.248109 4518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
W20260809 22:46:33.251178 3566 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 22:46:33.253340 3566 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d -> Peer 1e63d468dfbe4118851b84cb8dbefe4a (127.24.136.61:37453): Couldn't send request to peer 1e63d468dfbe4118851b84cb8dbefe4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.60.keytab.
Entry for principal oryx/127.24.136.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.60.keytab.
I20260809 22:46:33.255330 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.60:42669
--webserver_interface=127.24.136.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.60.keytab
--principal=oryx/127.24.136.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:42691,127.24.136.61:37453,127.24.136.60:42669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:33.259985 4355 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:33.260465 4355 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:33.260682 4355 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: stopping tablet replica
I20260809 22:46:33.268402 4355 master.cc:552] Master@127.24.136.61:37453 shutdown complete.
I20260809 22:46:33.268549 4355 master_runner.cc:305] Clearing existing system tablet
I20260809 22:46:33.269047 4355 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:33.269279 4355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:33.272322 4355 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 22:46:33.274025 4528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:33.274391 4355 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:33.274827 4355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:46:33.274971 4355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal
uuid: "1e63d468dfbe4118851b84cb8dbefe4a"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "0cd6543b064707ae3a67c058bee8adfd"
server_key_iv: "07288bbb671f8c24ece2e688e24321cc"
server_key_version: "encryptionkey@0"
I20260809 22:46:33.280133 4355 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:33.287623 4355 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 22:46:33.287988 4355 log.cc:1196] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 22:46:33.288776 4355 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Deleting consensus metadata
I20260809 22:46:33.289143 4355 master_runner.cc:315] Copying system tablet from 127.24.136.62:42691
I20260809 22:46:33.291790 4355 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: tablet copy: Beginning tablet copy session from remote peer at address 127.24.136.62:42691
I20260809 22:46:33.310781 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.292763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.24.136.62:42691)
0809 22:46:33.292953 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.292960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.296149 (+ 3189us) server_negotiation.cc:407] Connection header received
0809 22:46:33.296544 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.296551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.296657 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.296781 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.297898 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.299053 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.300863 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.301560 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.301693 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.304012 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.304019 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.304023 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.304067 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.307540 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.308366 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.308372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.308376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.308534 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.308980 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.308985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.308988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.309376 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.309533 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.310038 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.310414 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:46:33.310804 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.292732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44120)
0809 22:46:33.295998 (+ 3266us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.296022 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.296309 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.296831 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.296837 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.297400 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.297730 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.297744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.299230 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.299236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.300701 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.300717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.301731 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.301737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.301902 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.302639 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.302663 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.303760 (+ 1097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.307689 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.307695 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.307699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.308220 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.308651 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.308656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.308660 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.308853 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.309511 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.309516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.309745 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.309906 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.310237 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3087,"thread_start_us":2971,"threads_started":1}
I20260809 22:46:33.312114 3634 tablet_copy_service.cc:140] P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1e63d468dfbe4118851b84cb8dbefe4a ({username='oryx', principal='oryx/127.24.136.61@KRBTEST.COM'} at 127.0.0.1:44120)
I20260809 22:46:33.312433 3634 tablet_copy_service.cc:161] P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1e63d468dfbe4118851b84cb8dbefe4a at {username='oryx', principal='oryx/127.24.136.61@KRBTEST.COM'} at 127.0.0.1:44120: session id = 1e63d468dfbe4118851b84cb8dbefe4a-00000000000000000000000000000000
I20260809 22:46:33.316989 3634 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 22:46:33.319662 4355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 22:46:33.326865 4355 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: tablet copy: Starting download of 0 data blocks...
I20260809 22:46:33.327243 4355 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: tablet copy: Starting download of 1 WAL segments...
I20260809 22:46:33.330822 4355 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 22:46:33.335781 3634 tablet_copy_service.cc:342] P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Request end of tablet copy session 1e63d468dfbe4118851b84cb8dbefe4a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.136.61@KRBTEST.COM'} at 127.0.0.1:44120
I20260809 22:46:33.336041 3634 tablet_copy_service.cc:433] P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: ending tablet copy session 1e63d468dfbe4118851b84cb8dbefe4a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1e63d468dfbe4118851b84cb8dbefe4a
I20260809 22:46:33.347687 4355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:33.350373 4542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:33.360180 4548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:33.360886 4355 server_base.cc:1034] running on GCE node
W20260809 22:46:33.361110 4544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:33.361577 4355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:33.362248 4355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:33.363489 4355 hybrid_clock.cc:648] HybridClock initialized: now 1786315593363411 us; error 27 us; skew 500 ppm
I20260809 22:46:33.363732 4355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315593, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:33.365581 4355 init.cc:377] Logged in from keytab as oryx/127.24.136.61@KRBTEST.COM (short username oryx)
I20260809 22:46:33.368331 4355 webserver.cc:460] Webserver started at http://127.24.136.61:45885/ using document root <none> and password file <none>
I20260809 22:46:33.368659 4355 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:33.368736 4355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:33.371131 4355 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 22:46:33.372879 4558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:33.373317 4355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 22:46:33.373456 4355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal
uuid: "1e63d468dfbe4118851b84cb8dbefe4a"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "0cd6543b064707ae3a67c058bee8adfd"
server_key_iv: "07288bbb671f8c24ece2e688e24321cc"
server_key_version: "encryptionkey@0"
I20260809 22:46:33.373624 4355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:33.393885 4355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:33.397292 4355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:33.426101 4355 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.61:37453
I20260809 22:46:33.426152 4639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.61:37453 every 8 connection(s)
I20260809 22:46:33.427027 4355 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
I20260809 22:46:33.433362 4641 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:46:33.436436 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Bootstrap starting.
I20260809 22:46:33.450176 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:46:33.450835 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Bootstrap complete.
I20260809 22:46:33.452642 4641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } }
I20260809 22:46:33.452934 4641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1e63d468dfbe4118851b84cb8dbefe4a, State: Initialized, Role: LEARNER
I20260809 22:46:33.453161 4641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } }
I20260809 22:46:33.455338 4647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } } }
I20260809 22:46:33.455698 4641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:33.455925 4647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: This master's current role is: LEARNER
I20260809 22:46:33.463562 4666 catalog_manager.cc:1259] Loaded cluster ID: dd6b1a6cdddb4375a36414e5c98b8041
I20260809 22:46:33.463706 4666 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: loading cluster ID for follower catalog manager: success
I20260809 22:46:33.467568 4666 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: acquiring CA information for follower catalog manager: success
I20260809 22:46:33.470381 4666 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:46:33.547910 4345 heartbeater.cc:503] Master 127.24.136.62:42691 was elected leader, sending a full tablet report...
W20260809 22:46:33.672928 4526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:33.673265 4526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:33.673408 4526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:33.680992 4526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:33.681134 4526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:33.681221 4526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:33.681301 4526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:33.681367 4526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:33.692116 4526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:42691,127.24.136.61:37453,127.24.136.60:42669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.60.keytab
--principal=oryx/127.24.136.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.60:42669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.136.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:33.693611 4526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:33.694972 4526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:33.707315 4679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:33.707376 4682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:33.707315 4676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:33.708145 4526 server_base.cc:1034] running on GCE node
I20260809 22:46:33.708932 4526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:33.710492 4526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:33.711709 4526 hybrid_clock.cc:648] HybridClock initialized: now 1786315593711636 us; error 56 us; skew 500 ppm
I20260809 22:46:33.712060 4526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315593, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:33.715372 4526 init.cc:377] Logged in from keytab as oryx/127.24.136.60@KRBTEST.COM (short username oryx)
I20260809 22:46:33.716995 4526 webserver.cc:460] Webserver started at http://127.24.136.60:43027/ using document root <none> and password file <none>
I20260809 22:46:33.717702 4526 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:33.717815 4526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:33.718091 4526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:33.721517 4526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data/instance:
uuid: "06d41c5d02d542d9a6b1537b109a31ee"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "282dbdbcaab4e06897138a6dbf50b2e2"
server_key_iv: "317884cd061f64d1a1937e5d4baca3d8"
server_key_version: "encryptionkey@0"
I20260809 22:46:33.722318 4526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal/instance:
uuid: "06d41c5d02d542d9a6b1537b109a31ee"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "282dbdbcaab4e06897138a6dbf50b2e2"
server_key_iv: "317884cd061f64d1a1937e5d4baca3d8"
server_key_version: "encryptionkey@0"
I20260809 22:46:33.728139 4526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:46:33.731988 4691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:33.733484 4526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:33.733670 4526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal
uuid: "06d41c5d02d542d9a6b1537b109a31ee"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "282dbdbcaab4e06897138a6dbf50b2e2"
server_key_iv: "317884cd061f64d1a1937e5d4baca3d8"
server_key_version: "encryptionkey@0"
I20260809 22:46:33.733855 4526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:33.748116 4526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:33.752813 4526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:33.753083 4526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:33.778028 4526 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.60:42669
I20260809 22:46:33.778146 4784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.60:42669 every 8 connection(s)
I20260809 22:46:33.779945 4526 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
I20260809 22:46:33.781565 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 4526
I20260809 22:46:33.785461 4785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:33.796736 4774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.770159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.24.136.61:37453)
0809 22:46:33.777051 (+ 6892us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.777059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.777720 (+ 661us) server_negotiation.cc:407] Connection header received
0809 22:46:33.777939 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.777945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.778075 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.778217 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:33.778938 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.780509 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.792720 (+ 12211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.794159 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.794327 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.794602 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.796121 (+ 1519us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.796325 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6713,"thread_start_us":6574,"threads_started":1}
I20260809 22:46:33.797696 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.769986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.0.0.1:46106)
0809 22:46:33.777550 (+ 7564us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.777573 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.777783 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.778494 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.778500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.778531 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:33.778763 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.778776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.780708 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.780713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.790347 (+ 9634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.790373 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.794889 (+ 4516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.794897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.795149 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.795244 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.797129 (+ 1885us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.797339 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7422,"thread_start_us":7313,"threads_started":1}
I20260809 22:46:33.797461 4785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:46:33.799093 4598 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 22:46:33.799984 4647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b23e1a1756f4ebabb8c7d61ce3b1f9d. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } } }
I20260809 22:46:33.800251 4647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: This master's current role is: LEARNER
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315593, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.60@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.60@KRBTEST.COM
I20260809 22:46:33.825193 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.805121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44132 (local address 127.24.136.62:42691)
0809 22:46:33.805318 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.805324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.806224 (+ 900us) server_negotiation.cc:407] Connection header received
0809 22:46:33.807300 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.807306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.807381 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.807513 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.809462 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.810683 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.812896 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.813493 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.813657 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.818015 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.818020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.818024 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.818063 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.821439 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.822437 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.822444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.822448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.822637 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.823068 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.823074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.823078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.823430 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.823618 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.824588 (+ 970us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.824860 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:46:33.826393 4790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.792992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.24.136.60:42669)
0809 22:46:33.804055 (+ 11063us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.804081 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.804137 (+ 56us) server_negotiation.cc:407] Connection header received
0809 22:46:33.806886 (+ 2749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.806921 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.807278 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.807545 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.809497 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.810710 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.812671 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.813489 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.813694 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.818821 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.818836 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.818856 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.818899 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.822538 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.823345 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.823355 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.823358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.823554 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.823995 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.823999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.824001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.824409 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.824729 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.825145 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.825420 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10817,"thread_start_us":10569,"threads_started":1}
I20260809 22:46:33.826411 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.804758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44132)
0809 22:46:33.806038 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.806102 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.807022 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.807775 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.807782 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.808644 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.809236 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.809266 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.810846 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.810851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.812747 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.812760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.813763 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.813771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.813979 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.814705 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.814738 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.817776 (+ 3038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.821634 (+ 3858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.821651 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.821663 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.822247 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.822736 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.822741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.822744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.822930 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.823637 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.823647 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.824146 (+ 499us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.824421 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.825028 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"thread_start_us":747,"threads_started":1}
I20260809 22:46:33.828588 4785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315593, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.60@KRBTEST.COM for oryx/127.24.136.61@KRBTEST.COM
I20260809 22:46:33.841692 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.829283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.0.0.1:46116)
0809 22:46:33.829508 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.829528 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.829687 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.830117 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.830121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.830489 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.830656 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.830665 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.831789 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.831793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.832817 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.832832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.833618 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.833623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.833759 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.834294 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.834312 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.836467 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.839413 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.839418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.839421 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.839896 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.840273 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.840276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.840278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.840410 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.840897 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.840901 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.841048 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.841177 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.841444 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 22:46:33.841809 4774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.829493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.24.136.61:37453)
0809 22:46:33.829707 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.829714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.829769 (+ 55us) server_negotiation.cc:407] Connection header received
0809 22:46:33.829838 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.829843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.829923 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.830021 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.830810 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.831668 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.832979 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.833445 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.833541 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.836670 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.836675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.836679 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.836716 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.839269 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.839999 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.840003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.840006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.840185 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.840487 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.840489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.840491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.840768 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.840883 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.841325 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.841541 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:46:33.842832 4785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315593, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.60@KRBTEST.COM for oryx/127.24.136.60@KRBTEST.COM
I20260809 22:46:33.855352 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.843791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:42669 (local address 127.0.0.1:60380)
0809 22:46:33.843979 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.843996 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.844089 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.844331 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.844334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.844673 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.844824 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.844831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.845740 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.845743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.846588 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.846598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.847256 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.847260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.847576 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.848183 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.848202 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.850297 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.853107 (+ 2810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.853111 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.853114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.853483 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.853846 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.853849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.853851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.854013 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.854505 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.854508 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.854663 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.854796 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.855008 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 22:46:33.855558 4790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.843659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.24.136.60:42669)
0809 22:46:33.843849 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.843854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.844059 (+ 205us) server_negotiation.cc:407] Connection header received
0809 22:46:33.844168 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.844172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.844227 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.844306 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.844933 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.845634 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.846713 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.847122 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.847222 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.850517 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.850523 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.850526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.850558 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.852969 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.853610 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.853615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.853617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.853755 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.854101 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.854104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.854106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.854397 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.854509 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.854896 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.855162 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":58}
I20260809 22:46:33.860690 4785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: Bootstrap starting.
I20260809 22:46:33.863968 4785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:33.865020 4785 log.cc:824] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:33.867847 4785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: No bootstrap required, opened a new log
I20260809 22:46:33.875348 4785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } }
I20260809 22:46:33.875751 4785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:33.875845 4785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d41c5d02d542d9a6b1537b109a31ee, State: Initialized, Role: FOLLOWER
I20260809 22:46:33.876422 4785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } }
I20260809 22:46:33.877334 4794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } } }
I20260809 22:46:33.877660 4794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:33.878427 4785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:46:33.887814 4812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:46:33.887969 4812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:46:33.892805 4526 master_runner.cc:428] Detected that this master 06d41c5d02d542d9a6b1537b109a31ee is joining an existing cluster
I20260809 22:46:33.892931 4526 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.136.60:42669
I20260809 22:46:33.914707 4790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.897653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60396 (local address 127.24.136.60:42669)
0809 22:46:33.898271 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.898278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.898304 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:46:33.898521 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.898526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.898600 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.898676 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.900291 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.901513 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.903613 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.904452 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.904572 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.907441 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.907448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.907479 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.907524 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.911654 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.912423 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.912428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.912432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.912588 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.912994 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.913002 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.913006 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.913378 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.913518 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.914041 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.914319 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":431}
I20260809 22:46:33.915824 4820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.896996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:42669 (local address 127.0.0.1:60396)
0809 22:46:33.898160 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.898181 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.898351 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.898890 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.898894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.899714 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.900156 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.900170 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.901764 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.901769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.903482 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.903498 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.904870 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.904881 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.905079 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.905877 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.905901 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.907246 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.911821 (+ 4575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.911827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.911830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.912301 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.912720 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.912725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.912742 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.912899 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.913597 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.913602 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.913790 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.915001 (+ 1211us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.915339 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":25,"thread_start_us":855,"threads_started":1}
I20260809 22:46:33.915863 4774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.898866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.24.136.61:37453)
0809 22:46:33.899051 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.899057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.900068 (+ 1011us) server_negotiation.cc:407] Connection header received
0809 22:46:33.900332 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.900338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.900420 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.900491 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.902306 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.903392 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.905594 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.906277 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.906397 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.908997 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.909004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.909008 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.909053 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.912804 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.913544 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.913550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.913553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.913710 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.914152 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.914157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.914160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.914527 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.914647 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.915266 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.915534 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:46:33.916870 4827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.899190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.0.0.1:46120)
0809 22:46:33.899945 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.899970 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.900173 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.900765 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.900773 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.901529 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.902074 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.902091 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.903613 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.903617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.905405 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.905419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.906722 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.906728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.906897 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.907570 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.907596 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.908740 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.912943 (+ 4203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.912947 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.912950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.913404 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.913858 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.913862 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.913865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.914031 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.914756 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.914760 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.914954 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.916110 (+ 1156us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.916436 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":125,"thread_start_us":445,"threads_started":1}
I20260809 22:46:33.918936 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.896929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44140 (local address 127.24.136.62:42691)
0809 22:46:33.897142 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.897148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.898705 (+ 1557us) server_negotiation.cc:407] Connection header received
0809 22:46:33.898929 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.898935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.899018 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.899086 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.902229 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.903418 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.906498 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.907214 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.907325 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.910142 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.910150 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.910155 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.910202 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.913793 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.914620 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.914625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.914629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.915111 (+ 482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.915678 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.915683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.915686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.916503 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.916644 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.918290 (+ 1646us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.918568 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:46:33.920022 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.897309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44140)
0809 22:46:33.898612 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.898632 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.898794 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.899337 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.899341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.900449 (+ 1108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.901987 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.902007 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.903668 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.903673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.906280 (+ 2607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.906296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.907665 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.907674 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.907879 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.908603 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.908629 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.909895 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.913964 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.913970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.913974 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.914473 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.915288 (+ 815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.915294 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.915297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.915535 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.916747 (+ 1212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.916752 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.917254 (+ 502us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.919215 (+ 1961us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.919572 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1185,"mutex_wait_us":136,"thread_start_us":974,"threads_started":1}
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.62@KRBTEST.COM for oryx/127.24.136.60@KRBTEST.COM
I20260809 22:46:33.933452 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 3692
I20260809 22:46:33.940546 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.925542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:42669 (local address 127.0.0.1:60406)
0809 22:46:33.925737 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:33.925757 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:33.925869 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:33.926260 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:33.926264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:33.926664 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:33.926814 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.926822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.927906 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.927910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.928852 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:33.928865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.929628 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.929634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:33.929813 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.930455 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:33.930473 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:33.933511 (+ 3038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:33.937862 (+ 4351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.937869 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.937874 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.938339 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.938741 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:33.938746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:33.938750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:33.938980 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:33.939560 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:33.939566 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:33.939741 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 22:46:33.939869 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 22:46:33.940180 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 22:46:33.940550 4790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.925617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60406 (local address 127.24.136.60:42669)
0809 22:46:33.925799 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.925804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.925823 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:46:33.926017 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.926020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.926109 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.926194 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.926972 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.927785 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.929001 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.929477 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.929592 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.933773 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.933781 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.933790 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.933832 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.937581 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.938489 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.938494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.938496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.938617 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.939098 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.939104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.939108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.939419 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.939599 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.939986 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.940203 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:46:33.941987 3603 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.136.60:42669: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee"
member_type: NON_VOTER
last_known_addr {
host: "127.24.136.60"
port: 42669
}
attrs {
promote: true
}
}
dest_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d"
cas_config_opid_index: 5
I20260809 22:46:33.942957 3603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } }
I20260809 22:46:33.945144 4829 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:33.945960 4598 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 LEARNER]: Refusing update from remote peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 22:46:33.946699 4758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
Aug 09 22:46:33 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.1@KRBTEST.COM
I20260809 22:46:33.948630 4739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:33.949808 4829 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.136.60:42669
I20260809 22:46:33.960611 4758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:33.961035 4758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:33.971851 4526 master.cc:537] Master@127.24.136.60:42669 shutting down...
I20260809 22:46:33.972368 4598 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:33.973249 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:33.934540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.24.136.1:38659)
0809 22:46:33.935307 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:33.935317 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:33.935352 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:46:33.935430 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:33.935439 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:33.935841 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:33.936092 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:33.937878 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.939603 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.942095 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:33.943123 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:33.951612 (+ 8489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:33.951892 (+ 280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:33.951901 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:33.951918 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:33.951943 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:33.956029 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.957236 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.957247 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.957251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.957441 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:33.967017 (+ 9576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:33.967027 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:33.967031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:33.967709 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:33.968109 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:33.970996 (+ 2887us) server_negotiation.cc:299] Negotiation successful
0809 22:46:33.971362 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":382,"threads_started":1}
I20260809 22:46:33.976126 3892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:33.976838 4739 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Refusing update from remote peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 22:46:33.978542 4647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: NON_VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: true } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:33.978881 4647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: This master's current role is: LEARNER
I20260809 22:46:33.979961 4832 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:33.981745 4832 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: attempting to promote NON_VOTER 1e63d468dfbe4118851b84cb8dbefe4a to VOTER
I20260809 22:46:33.982678 4832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } }
I20260809 22:46:33.986129 4598 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 LEARNER]: Refusing update from remote peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 22:46:33.987306 4758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 22:46:33.990561 4758 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1e63d468dfbe4118851b84cb8dbefe4a (127.24.136.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:33.995550 4597 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1e63d468dfbe4118851b84cb8dbefe4a (127.24.136.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:33.996326 4526 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:33.996896 4526 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:33.997188 4526 tablet_replica.cc:330] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: stopping tablet replica
I20260809 22:46:34.000265 4829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:34.000677 4829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:34.001047 4829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:34.001271 4829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
W20260809 22:46:34.001667 3576 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d -> Peer 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60:42669): Couldn't send request to peer 06d41c5d02d542d9a6b1537b109a31ee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 22:46:34.001940 4647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:34.002233 4647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:34.008767 4526 master.cc:552] Master@127.24.136.60:42669 shutdown complete.
I20260809 22:46:34.008862 4526 master_runner.cc:305] Clearing existing system tablet
I20260809 22:46:34.009229 4526 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:34.009339 4526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:34.011725 4526 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 22:46:34.013190 4842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:34.013546 4526 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:34.013819 4526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 22:46:34.013924 4526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal
uuid: "06d41c5d02d542d9a6b1537b109a31ee"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "282dbdbcaab4e06897138a6dbf50b2e2"
server_key_iv: "317884cd061f64d1a1937e5d4baca3d8"
server_key_version: "encryptionkey@0"
I20260809 22:46:34.018010 4526 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:34.023603 4526 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 22:46:34.023779 4526 log.cc:1196] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 22:46:34.024216 4526 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: Deleting consensus metadata
I20260809 22:46:34.024448 4526 master_runner.cc:315] Copying system tablet from 127.24.136.62:42691
I20260809 22:46:34.026876 4526 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: tablet copy: Beginning tablet copy session from remote peer at address 127.24.136.62:42691
I20260809 22:46:34.044098 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.027721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.24.136.62:42691)
0809 22:46:34.027986 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:34.027993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:34.032186 (+ 4193us) server_negotiation.cc:407] Connection header received
0809 22:46:34.032401 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:34.032407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:34.032485 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:34.032574 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:34.033521 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.034251 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.035593 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.036090 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.036216 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.038295 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:34.038300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:34.038303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:34.038343 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:34.041199 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.041944 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.041950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.041952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.042100 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.042500 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.042504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.042506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.042819 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:34.042973 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:34.043451 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 22:46:34.043743 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260809 22:46:34.044420 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.027689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44144)
0809 22:46:34.032045 (+ 4356us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:34.032065 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:34.032262 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:34.032640 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:34.032645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:34.033120 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:34.033345 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.033355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.034420 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.034425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.035391 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.035405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.036285 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.036292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.036469 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.037170 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:34.037189 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:34.038065 (+ 876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:34.041379 (+ 3314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:34.041386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:34.041391 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:34.041813 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:34.042227 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:34.042233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:34.042237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:34.042388 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:34.042956 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:34.042962 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:34.043169 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 22:46:34.043331 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 22:46:34.043723 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4066,"thread_start_us":3936,"threads_started":1}
I20260809 22:46:34.045235 3634 tablet_copy_service.cc:140] P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 06d41c5d02d542d9a6b1537b109a31ee ({username='oryx', principal='oryx/127.24.136.60@KRBTEST.COM'} at 127.0.0.1:44144)
I20260809 22:46:34.045404 3634 tablet_copy_service.cc:161] P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 06d41c5d02d542d9a6b1537b109a31ee at {username='oryx', principal='oryx/127.24.136.60@KRBTEST.COM'} at 127.0.0.1:44144: session id = 06d41c5d02d542d9a6b1537b109a31ee-00000000000000000000000000000000
I20260809 22:46:34.047816 3634 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 22:46:34.049497 4526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 22:46:34.054437 4526 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: tablet copy: Starting download of 0 data blocks...
I20260809 22:46:34.054735 4526 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: tablet copy: Starting download of 1 WAL segments...
I20260809 22:46:34.057235 4526 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 22:46:34.060851 3634 tablet_copy_service.cc:342] P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Request end of tablet copy session 06d41c5d02d542d9a6b1537b109a31ee-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.136.60@KRBTEST.COM'} at 127.0.0.1:44144
I20260809 22:46:34.061043 3634 tablet_copy_service.cc:433] P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: ending tablet copy session 06d41c5d02d542d9a6b1537b109a31ee-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 06d41c5d02d542d9a6b1537b109a31ee
I20260809 22:46:34.071532 4526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:34.074345 4855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:34.074887 4856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:34.079702 4526 server_base.cc:1034] running on GCE node
W20260809 22:46:34.080111 4862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:34.080513 4526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:34.081156 4526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:34.082352 4526 hybrid_clock.cc:648] HybridClock initialized: now 1786315594082299 us; error 43 us; skew 500 ppm
I20260809 22:46:34.082576 4526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:34 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:34.084439 4526 init.cc:377] Logged in from keytab as oryx/127.24.136.60@KRBTEST.COM (short username oryx)
I20260809 22:46:34.085659 4526 webserver.cc:460] Webserver started at http://127.24.136.60:45629/ using document root <none> and password file <none>
I20260809 22:46:34.085935 4526 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:34.086038 4526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:34.088457 4526 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:46:34.090103 4874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:34.090520 4526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 22:46:34.090652 4526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal
uuid: "06d41c5d02d542d9a6b1537b109a31ee"
format_stamp: "Formatted at 2026-08-09 22:46:33 on dist-test-slave-pww0"
server_key: "282dbdbcaab4e06897138a6dbf50b2e2"
server_key_iv: "317884cd061f64d1a1937e5d4baca3d8"
server_key_version: "encryptionkey@0"
I20260809 22:46:34.090828 4526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:34.110339 4526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:34.113873 4526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:34.143545 4526 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.60:42669
I20260809 22:46:34.143589 4976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.60:42669 every 8 connection(s)
I20260809 22:46:34.144697 4526 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
I20260809 22:46:34.146992 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 3692
I20260809 22:46:34.150300 4977 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:46:34.154357 4977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: Bootstrap starting.
I20260809 22:46:34.172614 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:38659
--local_ip_for_outbound_sockets=127.24.136.1
--tserver_master_addrs=127.24.136.62:42691
--webserver_port=45857
--webserver_interface=127.24.136.1
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab
--principal=oryx/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.136.62:42691,127.24.136.61:37453
--tserver_master_addrs=127.24.136.62:42691,127.24.136.61:37453,127.24.136.60:42669 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:34.176229 4977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:46:34.176806 4977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: Bootstrap complete.
I20260809 22:46:34.177938 4977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } }
I20260809 22:46:34.178292 4977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d41c5d02d542d9a6b1537b109a31ee, State: Initialized, Role: LEARNER
I20260809 22:46:34.178562 4977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } }
I20260809 22:46:34.179531 4981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:34.179872 4977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:34.180152 4981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: This master's current role is: LEARNER
I20260809 22:46:34.191483 4998 catalog_manager.cc:1259] Loaded cluster ID: dd6b1a6cdddb4375a36414e5c98b8041
I20260809 22:46:34.191607 4998 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: loading cluster ID for follower catalog manager: success
I20260809 22:46:34.194378 4998 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: acquiring CA information for follower catalog manager: success
I20260809 22:46:34.196645 4998 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:46:34.375758 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.366494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:42669 (local address 127.0.0.1:60422)
0809 22:46:34.366815 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:34.366839 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:34.367017 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:34.368439 (+ 1422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:34.368446 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:34.368491 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:34.368751 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.368768 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.370879 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.370885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.372864 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.372889 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.374844 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.374851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.375052 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.375094 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:46:34.375225 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 22:46:34.375392 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260809 22:46:34.375841 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.367029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.24.136.60:42669)
0809 22:46:34.367956 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:34.367962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:34.367986 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:46:34.368073 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:34.368078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:34.368196 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:34.368353 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:34.368962 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.370658 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.373086 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.374627 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.374776 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.375062 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:34.375286 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 22:46:34.375451 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":619,"threads_started":1}
I20260809 22:46:34.376910 4926 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 22:46:34.377630 4981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b23e1a1756f4ebabb8c7d61ce3b1f9d. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:34.377948 4981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: This master's current role is: LEARNER
W20260809 22:46:34.554204 4984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:34.554508 4984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:34.554636 4984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:34.562438 4984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:34.562582 4984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:34.562734 4984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:34.573280 4984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab
--principal=oryx/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:38659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=45857
--tserver_master_addrs=127.24.136.62:42691,127.24.136.61:37453,127.24.136.60:42669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:34.574775 4984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:34.576229 4984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:34.590121 5016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:34.590169 5013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:34.590324 5012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:34.591125 4984 server_base.cc:1034] running on GCE node
I20260809 22:46:34.591871 4984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:34.592948 4984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:34.594195 4984 hybrid_clock.cc:648] HybridClock initialized: now 1786315594594129 us; error 52 us; skew 500 ppm
I20260809 22:46:34.594560 4984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:34 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:34.598070 4984 init.cc:377] Logged in from keytab as oryx/127.24.136.1@KRBTEST.COM (short username oryx)
I20260809 22:46:34.599761 4984 webserver.cc:460] Webserver started at http://127.24.136.1:45857/ using document root <none> and password file <none>
I20260809 22:46:34.600503 4984 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:34.600648 4984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:34.606784 4984 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 22:46:34.610102 5025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:34.611671 4984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:34.611848 4984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "1b0df5f2b5834104a810228d4ccd1052"
format_stamp: "Formatted at 2026-08-09 22:46:31 on dist-test-slave-pww0"
server_key: "b8c4af1363ef8594fcbe359f43ed99c4"
server_key_iv: "16d8eb14186720dd11a642c50aea96a8"
server_key_version: "encryptionkey@0"
I20260809 22:46:34.612895 4984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:34.637692 4984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:34.642485 4984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:34.642825 4984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:34.644062 4984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:34.645749 4984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:34.645872 4984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:34.645990 4984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:34.646062 4984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:34.679926 4984 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:38659
I20260809 22:46:34.680027 5202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:38659 every 8 connection(s)
I20260809 22:46:34.681598 4984 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:34.686430 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 4984
I20260809 22:46:34.686767 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 3908
Aug 09 22:46:34 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.60@KRBTEST.COM
Aug 09 22:46:34 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:34.723934 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.685269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:34529 (local address 127.24.136.60:42669)
0809 22:46:34.685814 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:34.685818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:34.698866 (+ 13048us) server_negotiation.cc:407] Connection header received
0809 22:46:34.701299 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:34.701307 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:34.701414 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:34.701517 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:34.704491 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.705715 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.708654 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.709320 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.709434 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.714979 (+ 5545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:34.714987 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:34.714993 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:34.715045 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:34.718757 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.720543 (+ 1786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.720549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.720553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.720760 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.721407 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.721414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.721417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.721804 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:34.721953 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:34.723206 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0809 22:46:34.723555 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409}
I20260809 22:46:34.726562 5216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.685547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:42669 (local address 127.24.136.1:34529)
0809 22:46:34.698646 (+ 13099us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:34.698702 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:34.700937 (+ 2235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:34.702024 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:34.702038 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:34.703082 (+ 1044us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:34.704194 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.704225 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.705954 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.705958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.708405 (+ 2447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.708426 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.709837 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.709844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.710191 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.711176 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:34.711213 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:34.714625 (+ 3412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:34.718969 (+ 4344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:34.718978 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:34.718997 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:34.719720 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:34.720922 (+ 1202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:34.720927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:34.720930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:34.721212 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:34.722196 (+ 984us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:34.722206 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:34.722718 (+ 512us) client_negotiation.cc:769] Sending connection context
0809 22:46:34.724187 (+ 1469us) client_negotiation.cc:240] Negotiation successful
0809 22:46:34.724899 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12759,"spinlock_wait_cycles":505088,"thread_start_us":10416,"threads_started":1}
I20260809 22:46:34.728602 5206 heartbeater.cc:347] Connected to a master server at 127.24.136.60:42669
I20260809 22:46:34.728971 5206 heartbeater.cc:464] Registering TS with master...
Aug 09 22:46:34 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.2@KRBTEST.COM
I20260809 22:46:34.730041 5206 heartbeater.cc:511] Master 127.24.136.60:42669 requested a full tablet report, sending...
I20260809 22:46:34.732539 5214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.685547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:48241)
0809 22:46:34.701840 (+ 16293us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:34.701860 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:34.701986 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:34.708829 (+ 6843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:34.708834 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:34.709537 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:34.709906 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.709918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.712779 (+ 2861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.712783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.714306 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.714327 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.717529 (+ 3202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.717536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.717714 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.719018 (+ 1304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:34.719056 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:34.722644 (+ 3588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:34.728266 (+ 5622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:34.728272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:34.728276 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:34.729092 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:34.729589 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:34.729594 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:34.729597 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:34.729774 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:34.730634 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:34.730639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:34.731551 (+ 912us) client_negotiation.cc:769] Sending connection context
0809 22:46:34.731719 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 22:46:34.732099 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16103,"mutex_wait_us":1535,"thread_start_us":14195,"threads_started":1}
I20260809 22:46:34.732623 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.684912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:48241 (local address 127.24.136.62:42691)
0809 22:46:34.695979 (+ 11067us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:34.695984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:34.705711 (+ 9727us) server_negotiation.cc:407] Connection header received
0809 22:46:34.705812 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:34.705818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:34.705921 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:34.706040 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:34.711385 (+ 5345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.712528 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.716271 (+ 3743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.717036 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.717151 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.722956 (+ 5805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:34.722964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:34.722969 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:34.723017 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:34.728082 (+ 5065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.729238 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.729244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.729248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.729435 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.729933 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.729938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.729941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.730408 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:34.730554 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:34.731906 (+ 1352us) server_negotiation.cc:299] Negotiation successful
0809 22:46:34.732235 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10936,"thread_start_us":10793,"threads_started":1}
I20260809 22:46:34.733032 4905 ts_manager.cc:194] Registered new tserver with Master: 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1:38659)
I20260809 22:46:34.733731 5204 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42691
I20260809 22:46:34.733881 5204 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:34.734354 5204 heartbeater.cc:511] Master 127.24.136.62:42691 requested a full tablet report, sending...
Aug 09 22:46:34 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.61@KRBTEST.COM
I20260809 22:46:34.735653 3603 ts_manager.cc:194] Re-registered known tserver with Master: 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1:38659)
I20260809 22:46:34.736718 3603 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:48241
I20260809 22:46:34.738848 5217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.688216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.24.136.2:45875)
0809 22:46:34.715971 (+ 27755us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:34.715983 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:34.716020 (+ 37us) server_negotiation.cc:407] Connection header received
0809 22:46:34.716108 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:34.716119 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:34.716423 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:34.716601 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:34.719251 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.720896 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.722866 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.723859 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.723995 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.730304 (+ 6309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:34.730324 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:34.730341 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:34.730394 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:34.735292 (+ 4898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.736267 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.736278 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.736281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.736428 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.736919 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.736925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.736929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.737411 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:34.737776 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:34.738125 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 22:46:34.738426 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27528,"thread_start_us":32006,"threads_started":1}
I20260809 22:46:34.740327 4122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:34.742651 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.702270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.24.136.1:49551)
0809 22:46:34.720287 (+ 18017us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:34.720306 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:34.720423 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:34.723832 (+ 3409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:34.723837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:34.724521 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:34.724889 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.724903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.726556 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.726562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.728130 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:34.728147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.730224 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.730250 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:34.730468 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.731100 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:34.731122 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:34.735890 (+ 4768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:34.739953 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:34.739958 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:34.739963 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:34.740407 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:34.740786 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:34.740792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:34.740796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:34.740945 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:34.741574 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:34.741579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:34.741752 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 22:46:34.741892 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 22:46:34.742229 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17883,"thread_start_us":17713,"threads_started":1}
I20260809 22:46:34.743721 5222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.702268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:49551 (local address 127.24.136.61:37453)
0809 22:46:34.723002 (+ 20734us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:34.723010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:34.723039 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:46:34.723105 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:34.723110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:34.723262 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:34.723400 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:34.725110 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.726353 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.728362 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.729900 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.730823 (+ 923us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.736180 (+ 5357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:34.736188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:34.736193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:34.736240 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:34.739808 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.740528 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.740534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.740537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.740671 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.741063 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.741068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.741072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.741432 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:34.741572 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:34.743070 (+ 1498us) server_negotiation.cc:299] Negotiation successful
0809 22:46:34.743315 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20581,"thread_start_us":21330,"threads_started":1}
I20260809 22:46:34.744573 5208 heartbeater.cc:347] Connected to a master server at 127.24.136.61:37453
I20260809 22:46:34.744732 5208 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:34.745132 5208 heartbeater.cc:511] Master 127.24.136.61:37453 requested a full tablet report, sending...
I20260809 22:46:34.746914 4582 ts_manager.cc:194] Registered new tserver with Master: 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1:38659)
I20260809 22:46:34.827768 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 3908
I20260809 22:46:34.827739 4758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: NON_VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: true } } }
I20260809 22:46:34.837134 4758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:34.838281 5225 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: attempting to promote NON_VOTER 06d41c5d02d542d9a6b1537b109a31ee to VOTER
I20260809 22:46:34.839076 5225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } }
I20260809 22:46:34.841463 4598 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Refusing update from remote peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 22:46:34.841717 4926 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 LEARNER]: Refusing update from remote peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 22:46:34.842192 4758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 22:46:34.842620 5226 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 22:46:34.845721 4758 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:34.846232 4598 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:34.848838 4758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:34.849207 4758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:34.849058 5226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:34.849352 5226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:34.851038 4758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:34.851411 4758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:34.851969 4926 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:34.855271 4981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:34.855602 4981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:34.855885 5229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:34.856315 5229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:34.858449 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:45875
--local_ip_for_outbound_sockets=127.24.136.2
--tserver_master_addrs=127.24.136.62:42691
--webserver_port=37281
--webserver_interface=127.24.136.2
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab
--principal=oryx/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.136.62:42691,127.24.136.61:37453
--tserver_master_addrs=127.24.136.62:42691,127.24.136.61:37453,127.24.136.60:42669 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:35.219921 5230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:35.220244 5230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:35.220347 5230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:35.228236 5230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:35.228367 5230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:35.228485 5230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:35.238889 5230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab
--principal=oryx/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:45875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=37281
--tserver_master_addrs=127.24.136.62:42691,127.24.136.61:37453,127.24.136.60:42669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:35.240342 5230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:35.241674 5230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:35.255533 5239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:35.255548 5242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:35.255573 5238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:35.256647 5230 server_base.cc:1034] running on GCE node
I20260809 22:46:35.257429 5230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:35.258476 5230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:35.259696 5230 hybrid_clock.cc:648] HybridClock initialized: now 1786315595259624 us; error 50 us; skew 500 ppm
I20260809 22:46:35.260063 5230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:35 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:35.263906 5230 init.cc:377] Logged in from keytab as oryx/127.24.136.2@KRBTEST.COM (short username oryx)
I20260809 22:46:35.265821 5230 webserver.cc:460] Webserver started at http://127.24.136.2:37281/ using document root <none> and password file <none>
I20260809 22:46:35.266628 5230 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:35.266789 5230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:35.273253 5230 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 22:46:35.276669 5254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:35.278386 5230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:46:35.278581 5230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "88bc57d6506741ff97979bf46d4b9959"
format_stamp: "Formatted at 2026-08-09 22:46:31 on dist-test-slave-pww0"
server_key: "b218ba34ca1188ef6ad2e97df8f14f21"
server_key_iv: "2c6aa4d48e0c94fb66f6b00b5db1058c"
server_key_version: "encryptionkey@0"
I20260809 22:46:35.279693 5230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:35.296509 5230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:35.301126 5230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:35.301440 5230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:35.302574 5230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:35.304334 5230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:35.304455 5230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:35.304574 5230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:35.304669 5230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:35.337411 5230 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:45875
I20260809 22:46:35.337468 5432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:45875 every 8 connection(s)
I20260809 22:46:35.338992 5230 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:35.345588 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 5230
I20260809 22:46:35.345896 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 4137
Aug 09 22:46:35 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.61@KRBTEST.COM
Aug 09 22:46:35 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
Aug 09 22:46:35 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.3@KRBTEST.COM
Aug 09 22:46:35 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.60@KRBTEST.COM
I20260809 22:46:35.369977 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.342272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:34943 (local address 127.24.136.62:42691)
0809 22:46:35.342875 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:35.342881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:35.344848 (+ 1967us) server_negotiation.cc:407] Connection header received
0809 22:46:35.348272 (+ 3424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:35.348279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:35.348374 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:35.348483 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:35.351487 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.352618 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.355091 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.355826 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.355944 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:35.361801 (+ 5857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:35.361807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:35.361811 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:35.361848 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:35.366280 (+ 4432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:35.367174 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:35.367181 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:35.367185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:35.367365 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:35.367872 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:35.367877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:35.367881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:35.368613 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:35.368742 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:35.369355 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 22:46:35.369611 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":345,"threads_started":1}
I20260809 22:46:35.371331 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.345145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:49791 (local address 127.24.136.61:37453)
0809 22:46:35.346694 (+ 1549us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:35.346699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:35.347218 (+ 519us) server_negotiation.cc:407] Connection header received
0809 22:46:35.348434 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:35.348952 (+ 518us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:35.349065 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:35.349175 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:35.351486 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.352391 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.355128 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.355665 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.355797 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:35.360966 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:35.360973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:35.360978 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:35.361170 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:35.365373 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:35.366196 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:35.366202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:35.366205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:35.366379 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:35.367023 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:35.367028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:35.367031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:35.367427 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:35.367609 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:35.370667 (+ 3058us) server_negotiation.cc:299] Negotiation successful
0809 22:46:35.370970 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1453,"thread_start_us":1325,"threads_started":1}
I20260809 22:46:35.371712 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.342446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:33969 (local address 127.24.136.60:42669)
0809 22:46:35.343083 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:35.343090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:35.344391 (+ 1301us) server_negotiation.cc:407] Connection header received
0809 22:46:35.348490 (+ 4099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:35.348496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:35.348578 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:35.348681 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:35.352912 (+ 4231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.354172 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.358138 (+ 3966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.358818 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.358948 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:35.363583 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:35.363591 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:35.363596 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:35.363643 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:35.367721 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:35.368500 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:35.368506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:35.368510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:35.368698 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:35.369156 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:35.369161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:35.369163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:35.369501 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:35.369646 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:35.371066 (+ 1420us) server_negotiation.cc:299] Negotiation successful
0809 22:46:35.371320 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":360,"threads_started":1}
I20260809 22:46:35.371970 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.347416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35432 (local address 127.24.136.3:44141)
0809 22:46:35.348047 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:35.348056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:35.348087 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:46:35.348168 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:35.348178 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:35.348458 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:35.348633 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:35.351200 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.352888 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.354797 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.355763 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.355892 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:35.363709 (+ 7817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:35.363729 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:35.363749 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:35.363812 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:35.367500 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:35.368440 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:35.368450 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:35.368453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:35.368630 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:35.369132 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:35.369138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:35.369141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:35.369588 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:35.369885 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:35.371205 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0809 22:46:35.371539 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":282,"threads_started":1}
I20260809 22:46:35.372097 5448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.343109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:42669 (local address 127.24.136.2:33969)
0809 22:46:35.344132 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:35.344196 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:35.347976 (+ 3780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:35.348794 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:35.348805 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:35.350032 (+ 1227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:35.351243 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.351273 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.354357 (+ 3084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.354362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.357906 (+ 3544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.357924 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.359209 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.359216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.359395 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:35.360021 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:35.360045 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:35.363303 (+ 3258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:35.367863 (+ 4560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:35.367867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:35.367871 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:35.368335 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:35.368829 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:35.368835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:35.368838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:35.369004 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:35.369709 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:35.369714 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:35.369854 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 22:46:35.369981 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 22:46:35.370298 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":92,"spinlock_wait_cycles":24064,"thread_start_us":379,"threads_started":1}
I20260809 22:46:35.372298 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.343109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.2:34943)
0809 22:46:35.344724 (+ 1615us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:35.344739 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:35.347986 (+ 3247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:35.348790 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:35.348802 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:35.350032 (+ 1230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:35.351245 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.351273 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.352800 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.352805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.354941 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.354960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.356007 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.356014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.356279 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:35.357064 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:35.357105 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:35.361568 (+ 4463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:35.366455 (+ 4887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:35.366459 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:35.366463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:35.367016 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:35.367556 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:35.367561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:35.367565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:35.367735 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:35.368838 (+ 1103us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:35.368843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:35.369020 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 22:46:35.370480 (+ 1460us) client_negotiation.cc:240] Negotiation successful
0809 22:46:35.370766 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1481,"mutex_wait_us":224,"spinlock_wait_cycles":52224,"thread_start_us":1202,"threads_started":1}
I20260809 22:46:35.372071 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.345105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.24.136.2:49791)
0809 22:46:35.347026 (+ 1921us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:35.347040 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:35.348308 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:35.349370 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:35.349378 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:35.350034 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:35.351245 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.351273 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.352556 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.352561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.354941 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.354960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.356641 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.356653 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.356842 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:35.357360 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:35.357382 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:35.360681 (+ 3299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:35.365535 (+ 4854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:35.365546 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:35.365564 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:35.366055 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:35.366543 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:35.366548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:35.366552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:35.366861 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:35.367719 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:35.367728 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:35.368211 (+ 483us) client_negotiation.cc:769] Sending connection context
0809 22:46:35.368580 (+ 369us) client_negotiation.cc:240] Negotiation successful
0809 22:46:35.369192 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1821,"mutex_wait_us":256,"spinlock_wait_cycles":6400,"thread_start_us":1712,"threads_started":1}
I20260809 22:46:35.373512 4337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:35.374186 5441 heartbeater.cc:347] Connected to a master server at 127.24.136.61:37453
I20260809 22:46:35.374279 5433 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42691
I20260809 22:46:35.374569 5433 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:35.374630 5434 heartbeater.cc:347] Connected to a master server at 127.24.136.60:42669
I20260809 22:46:35.374742 5434 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:35.374574 5441 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:35.375802 5433 heartbeater.cc:511] Master 127.24.136.62:42691 requested a full tablet report, sending...
I20260809 22:46:35.376072 5441 heartbeater.cc:511] Master 127.24.136.61:37453 requested a full tablet report, sending...
I20260809 22:46:35.377281 5434 heartbeater.cc:511] Master 127.24.136.60:42669 requested a full tablet report, sending...
I20260809 22:46:35.377926 3603 ts_manager.cc:194] Re-registered known tserver with Master: 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875)
I20260809 22:46:35.378199 4582 ts_manager.cc:194] Registered new tserver with Master: 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875)
I20260809 22:46:35.378569 4905 ts_manager.cc:194] Registered new tserver with Master: 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875)
I20260809 22:46:35.379182 3603 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:34943
I20260809 22:46:35.466585 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 4137
I20260809 22:46:35.486850 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:44141
--local_ip_for_outbound_sockets=127.24.136.3
--tserver_master_addrs=127.24.136.62:42691
--webserver_port=46255
--webserver_interface=127.24.136.3
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab
--principal=oryx/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.136.62:42691,127.24.136.61:37453
--tserver_master_addrs=127.24.136.62:42691,127.24.136.61:37453,127.24.136.60:42669 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:35.739713 5204 heartbeater.cc:503] Master 127.24.136.62:42691 was elected leader, sending a full tablet report...
W20260809 22:46:35.846675 5457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:35.847013 5457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:35.847158 5457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:35.855933 5457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:35.856073 5457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:35.856238 5457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:35.867395 5457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab
--principal=oryx/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:44141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=46255
--tserver_master_addrs=127.24.136.62:42691,127.24.136.61:37453,127.24.136.60:42669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:35.868928 5457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:35.870369 5457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:35.885409 5464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:35.885442 5465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:35.885401 5467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:35.886646 5457 server_base.cc:1034] running on GCE node
I20260809 22:46:35.887387 5457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:35.888561 5457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:35.889811 5457 hybrid_clock.cc:648] HybridClock initialized: now 1786315595889750 us; error 61 us; skew 500 ppm
I20260809 22:46:35.890208 5457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:35 dist-test-slave-pww0 krb5kdc[3516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:35.894035 5457 init.cc:377] Logged in from keytab as oryx/127.24.136.3@KRBTEST.COM (short username oryx)
I20260809 22:46:35.895716 5457 webserver.cc:460] Webserver started at http://127.24.136.3:46255/ using document root <none> and password file <none>
I20260809 22:46:35.896472 5457 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:35.896597 5457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:35.903211 5457 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 22:46:35.906873 5477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:35.908528 5457 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:46:35.908738 5457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "71a825ab55274756b03f4c319c77286a"
format_stamp: "Formatted at 2026-08-09 22:46:32 on dist-test-slave-pww0"
server_key: "de83d754d2663cfe702254d4637f4f4c"
server_key_iv: "3fc377d74e7d97ef5252ba1b00c25fcb"
server_key_version: "encryptionkey@0"
I20260809 22:46:35.909866 5457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:35.933231 5457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:35.938022 5457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:35.938313 5457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:35.939410 5457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:35.941119 5457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:35.941221 5457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:35.941344 5457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:35.941417 5457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:35.975555 5457 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:44141
I20260809 22:46:35.975674 5650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:44141 every 8 connection(s)
I20260809 22:46:35.977172 5457 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:35.984560 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 5457
Aug 09 22:46:35 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
Aug 09 22:46:35 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.61@KRBTEST.COM
Aug 09 22:46:36 dist-test-slave-pww0 krb5kdc[3516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.60@KRBTEST.COM
I20260809 22:46:36.016423 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.981417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.24.136.3:53357)
0809 22:46:35.982608 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:35.982692 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:35.985658 (+ 2966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:35.987053 (+ 1395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:35.987071 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:35.988211 (+ 1140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:35.989633 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.989670 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.993130 (+ 3460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.993137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.996007 (+ 2870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.996026 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.997415 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.997422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.997626 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:35.998270 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:35.998294 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:36.001814 (+ 3520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:36.008710 (+ 6896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:36.008723 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:36.008746 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:36.009396 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:36.011946 (+ 2550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:36.011952 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:36.011956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:36.012317 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:36.013173 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:36.013184 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:36.013757 (+ 573us) client_negotiation.cc:769] Sending connection context
0809 22:46:36.014169 (+ 412us) client_negotiation.cc:240] Negotiation successful
0809 22:46:36.014813 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"mutex_wait_us":24,"spinlock_wait_cycles":86400,"thread_start_us":489,"threads_started":1}
I20260809 22:46:36.018128 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.980638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53357 (local address 127.24.136.61:37453)
0809 22:46:35.981817 (+ 1179us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:35.981822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:35.982848 (+ 1026us) server_negotiation.cc:407] Connection header received
0809 22:46:35.986134 (+ 3286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:35.986142 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:35.986264 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:35.986363 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:35.991530 (+ 5167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.992852 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.996229 (+ 3377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.996966 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.997073 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.002313 (+ 5240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.002322 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.002327 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.002385 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.008445 (+ 6060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.009551 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.011545 (+ 1994us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.011550 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.011765 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.012495 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.012501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.012504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.012904 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.013059 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.017425 (+ 4366us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.017699 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1064,"thread_start_us":948,"threads_started":1}
I20260809 22:46:36.020900 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.983429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.3:49707)
0809 22:46:35.984185 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:35.984202 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:35.985679 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:35.988527 (+ 2848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:35.988532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:35.989308 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:35.989731 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.989742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.991670 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.991675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.994081 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.994103 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.995400 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.995415 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:35.995820 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:35.996785 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:35.996830 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:36.002030 (+ 5200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:36.017195 (+ 15165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:36.017201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:36.017205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:36.017825 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:36.018704 (+ 879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:36.018710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:36.018714 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:36.018900 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:36.019720 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:36.019726 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:36.019921 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 22:46:36.020086 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 22:46:36.020437 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":125,"thread_start_us":397,"threads_started":1}
I20260809 22:46:36.021130 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.991918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60424 (local address 127.24.136.60:42669)
0809 22:46:36.000036 (+ 8118us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.000044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.000670 (+ 626us) server_negotiation.cc:407] Connection header received
0809 22:46:36.000862 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.000867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.000969 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.001076 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:36.003440 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.004840 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.006954 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.007674 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.007814 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.012687 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.012695 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.012699 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.012746 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.016400 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.017529 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.017535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.017538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.017788 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.018267 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.018331 (+ 64us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.018334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.018718 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.018955 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.020373 (+ 1418us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.020773 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7941,"thread_start_us":7806,"threads_started":1}
I20260809 22:46:36.021123 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.983583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:49707 (local address 127.24.136.62:42691)
0809 22:46:35.988129 (+ 4546us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:35.988137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:35.988163 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:46:35.988236 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:35.988243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:35.988337 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:35.988485 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:35.989929 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.991232 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.994301 (+ 3069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.994990 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.995149 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.002310 (+ 7161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.002319 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.002325 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.002384 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.011379 (+ 8995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.018393 (+ 7014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.018399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.018402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.018584 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.019135 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.019140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.019143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.019596 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.019756 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.020453 (+ 697us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.020749 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4399,"thread_start_us":4257,"threads_started":1}
I20260809 22:46:36.023864 5651 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42691
I20260809 22:46:36.024261 5651 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:36.025645 5651 heartbeater.cc:511] Master 127.24.136.62:42691 requested a full tablet report, sending...
I20260809 22:46:36.027873 3603 ts_manager.cc:194] Re-registered known tserver with Master: 71a825ab55274756b03f4c319c77286a (127.24.136.3:44141)
I20260809 22:46:36.028347 5656 heartbeater.cc:347] Connected to a master server at 127.24.136.61:37453
I20260809 22:46:36.028507 5656 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:36.028903 3603 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:49707
I20260809 22:46:36.028961 5656 heartbeater.cc:511] Master 127.24.136.61:37453 requested a full tablet report, sending...
I20260809 22:46:36.030499 4582 ts_manager.cc:194] Registered new tserver with Master: 71a825ab55274756b03f4c319c77286a (127.24.136.3:44141)
I20260809 22:46:36.030759 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.980597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:58831 (local address 127.24.136.60:42669)
0809 22:46:35.981156 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:35.981163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:35.982980 (+ 1817us) server_negotiation.cc:407] Connection header received
0809 22:46:35.986253 (+ 3273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:35.986257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:35.986359 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:35.986504 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:35.989939 (+ 3435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.990908 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.003905 (+ 12997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.004641 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.004751 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.021561 (+ 16810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.021568 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.021573 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.021621 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.025596 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.026934 (+ 1338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.026939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.026943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.027121 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.027681 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.027686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.027690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.028062 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.028203 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.030074 (+ 1871us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.030352 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":301,"threads_started":1}
I20260809 22:46:36.032368 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.981418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:42669 (local address 127.24.136.3:58831)
0809 22:46:35.982608 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:35.982692 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:35.985681 (+ 2989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:35.987202 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:35.987206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:35.988211 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:35.989631 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:35.989667 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:35.991322 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:35.991328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.003697 (+ 12369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.003715 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.005119 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.005126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.005333 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.006201 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:36.006239 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:36.020020 (+ 13781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:36.026148 (+ 6128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:36.026154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:36.026157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:36.026785 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:36.027283 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:36.027287 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:36.027290 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:36.027534 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:36.029231 (+ 1697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:36.029239 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:36.029417 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 22:46:36.031489 (+ 2072us) client_negotiation.cc:240] Negotiation successful
0809 22:46:36.031912 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":143,"spinlock_wait_cycles":14464,"thread_start_us":412,"threads_started":1}
I20260809 22:46:36.033288 5655 heartbeater.cc:347] Connected to a master server at 127.24.136.60:42669
I20260809 22:46:36.033414 5655 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:36.033802 5655 heartbeater.cc:511] Master 127.24.136.60:42669 requested a full tablet report, sending...
I20260809 22:46:36.034922 4905 ts_manager.cc:194] Registered new tserver with Master: 71a825ab55274756b03f4c319c77286a (127.24.136.3:44141)
I20260809 22:46:36.039611 5677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:35.991333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.24.136.62:42691)
0809 22:46:36.006026 (+ 14693us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.006032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.010177 (+ 4145us) server_negotiation.cc:407] Connection header received
0809 22:46:36.010267 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.010291 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.010399 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.010518 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:36.013062 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.014760 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.018071 (+ 3311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.019539 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.021839 (+ 2300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.033097 (+ 11258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.033105 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.033110 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.033155 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.036861 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.037639 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.037646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.037650 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.037802 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.038184 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.038188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.038190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.038467 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.038659 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.039077 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.039261 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14533,"thread_start_us":14386,"threads_started":1}
I20260809 22:46:36.044374 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.010117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.24.136.61:37453)
0809 22:46:36.018855 (+ 8738us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.018861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.024286 (+ 5425us) server_negotiation.cc:407] Connection header received
0809 22:46:36.024368 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.024373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.024476 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.024579 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:36.026902 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.028603 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.034204 (+ 5601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.034889 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.035022 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.038211 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.038216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.038220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.038258 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.041587 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.042292 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.042298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.042302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.042444 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.042844 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.042850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.042854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.043158 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.043430 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.043755 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.044007 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8614,"thread_start_us":17340,"threads_started":1}
I20260809 22:46:36.044400 3603 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:46:36.051450 3603 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:36.090878 5706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.074532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:38659 (local address 127.0.0.1:36300)
0809 22:46:36.081829 (+ 7297us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:36.081845 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:36.081985 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:36.082743 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:36.082748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:36.082777 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:36.083229 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.083243 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.085483 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.085488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.087725 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.087738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.089999 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.090006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.090179 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.090217 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:46:36.090313 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:46:36.090461 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7190,"thread_start_us":7081,"threads_started":1}
I20260809 22:46:36.092403 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.074744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.24.136.1:38659)
0809 22:46:36.081434 (+ 6690us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.081443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.081967 (+ 524us) server_negotiation.cc:407] Connection header received
0809 22:46:36.082154 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.082164 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.082484 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.083498 (+ 1014us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:36.083791 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.085316 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.087999 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.089736 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.091185 (+ 1449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.091723 (+ 538us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.091808 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.092006 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6508,"thread_start_us":6388,"threads_started":1}
I20260809 22:46:36.094580 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.073885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44141 (local address 127.0.0.1:35436)
0809 22:46:36.081176 (+ 7291us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:36.081200 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:36.082183 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:36.083604 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:36.083608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:36.083634 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:36.083999 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.084012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.087709 (+ 3697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.087715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.089952 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.089967 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.093729 (+ 3762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.093737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.093937 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.093973 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:46:36.094075 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 22:46:36.094230 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7120,"thread_start_us":374,"threads_started":1}
I20260809 22:46:36.094641 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.074285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.24.136.3:44141)
0809 22:46:36.082855 (+ 8570us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.082864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.082899 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:46:36.083000 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.083007 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.083252 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.085460 (+ 2208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:36.085784 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.087526 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.092004 (+ 4478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.093484 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.093638 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.093893 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.094149 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.094323 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8380,"thread_start_us":12667,"threads_started":1}
I20260809 22:46:36.097512 5097 tablet_service.cc:1467] Processing CreateTablet for tablet e7fbb6764abd446f9c0bd49216ccf86a (DEFAULT_TABLE table=test-table [id=94afbf7f9f5e4e22bfc91e68c08022ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:36.099239 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7fbb6764abd446f9c0bd49216ccf86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:36.098831 5552 tablet_service.cc:1467] Processing CreateTablet for tablet e7fbb6764abd446f9c0bd49216ccf86a (DEFAULT_TABLE table=test-table [id=94afbf7f9f5e4e22bfc91e68c08022ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:36.100666 5552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7fbb6764abd446f9c0bd49216ccf86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:36.105057 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.082020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:45875 (local address 127.0.0.1:60042)
0809 22:46:36.090957 (+ 8937us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:36.090982 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:36.091149 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:36.097129 (+ 5980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:36.097135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:36.097168 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:36.097736 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.097752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.099816 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.099822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.101916 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.101944 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.104046 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.104055 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.104324 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.104364 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 22:46:36.104479 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 22:46:36.104641 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8733,"thread_start_us":9325,"threads_started":1}
I20260809 22:46:36.105844 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.082318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.24.136.2:45875)
0809 22:46:36.094927 (+ 12609us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.094936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.094969 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:46:36.096374 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.096383 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.096644 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.096892 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:36.097989 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.099548 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.102187 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.103764 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.103936 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.104155 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.105383 (+ 1228us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.105572 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12391,"thread_start_us":13679,"threads_started":1}
I20260809 22:46:36.110446 5334 tablet_service.cc:1467] Processing CreateTablet for tablet e7fbb6764abd446f9c0bd49216ccf86a (DEFAULT_TABLE table=test-table [id=94afbf7f9f5e4e22bfc91e68c08022ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:36.112100 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7fbb6764abd446f9c0bd49216ccf86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:36.114373 5712 tablet_bootstrap.cc:492] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: Bootstrap starting.
I20260809 22:46:36.116803 5713 tablet_bootstrap.cc:492] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: Bootstrap starting.
I20260809 22:46:36.118503 5712 tablet_bootstrap.cc:654] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:36.119982 5712 log.cc:824] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:36.120744 5713 tablet_bootstrap.cc:654] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:36.121833 5713 log.cc:824] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:36.123436 5712 tablet_bootstrap.cc:492] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: No bootstrap required, opened a new log
I20260809 22:46:36.123770 5712 ts_tablet_manager.cc:1397] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260809 22:46:36.125056 5713 tablet_bootstrap.cc:492] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: No bootstrap required, opened a new log
I20260809 22:46:36.125347 5713 ts_tablet_manager.cc:1397] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:46:36.135102 5717 tablet_bootstrap.cc:492] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: Bootstrap starting.
I20260809 22:46:36.134866 5712 raft_consensus.cc:348] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:36.135440 5712 raft_consensus.cc:374] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:36.135586 5712 raft_consensus.cc:729] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71a825ab55274756b03f4c319c77286a, State: Initialized, Role: FOLLOWER
I20260809 22:46:36.136247 5713 raft_consensus.cc:348] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:36.136452 5712 consensus_queue.cc:260] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:36.136744 5713 raft_consensus.cc:374] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:36.136847 5713 raft_consensus.cc:729] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0df5f2b5834104a810228d4ccd1052, State: Initialized, Role: FOLLOWER
I20260809 22:46:36.137620 5713 consensus_queue.cc:260] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:36.138334 5651 heartbeater.cc:503] Master 127.24.136.62:42691 was elected leader, sending a full tablet report...
I20260809 22:46:36.139074 5712 ts_tablet_manager.cc:1428] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260809 22:46:36.139555 5717 tablet_bootstrap.cc:654] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:36.142961 5713 ts_tablet_manager.cc:1428] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260809 22:46:36.145152 5717 log.cc:824] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:36.152463 5717 tablet_bootstrap.cc:492] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: No bootstrap required, opened a new log
I20260809 22:46:36.152803 5717 ts_tablet_manager.cc:1397] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.004s
I20260809 22:46:36.161206 5717 raft_consensus.cc:348] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:36.161680 5717 raft_consensus.cc:374] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:36.161815 5717 raft_consensus.cc:729] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bc57d6506741ff97979bf46d4b9959, State: Initialized, Role: FOLLOWER
I20260809 22:46:36.162494 5717 consensus_queue.cc:260] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:36.163779 5433 heartbeater.cc:503] Master 127.24.136.62:42691 was elected leader, sending a full tablet report...
I20260809 22:46:36.164609 5717 ts_tablet_manager.cc:1428] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260809 22:46:36.181569 5718 raft_consensus.cc:482] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:36.181912 5718 raft_consensus.cc:504] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:36.183799 5718 leader_election.cc:288] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1:38659), 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875)
I20260809 22:46:36.191674 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.184565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:43955 (local address 127.24.136.2:45875)
0809 22:46:36.184766 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.184772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.184791 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:46:36.184857 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.184862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.184930 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.185013 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:36.185381 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.186463 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.189164 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.190438 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.190607 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.190751 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.191221 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.191368 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:46:36.191761 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.184002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:38659 (local address 127.24.136.3:38961)
0809 22:46:36.184228 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:36.184249 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:36.184389 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:36.184833 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:36.184837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:36.184869 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:36.185090 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.185102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.186619 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.186650 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.188962 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.188982 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.190669 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.190676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.191066 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.191094 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:46:36.191188 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 22:46:36.191403 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":150}
W20260809 22:46:36.192034 5212 tablet.cc:2369] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:36.192131 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.184399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:45875 (local address 127.24.136.3:43955)
0809 22:46:36.184677 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:36.184691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:36.184768 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:36.185070 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:36.185075 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:36.185096 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:36.185261 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.185269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.186619 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.186626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.188962 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.188982 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.190669 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.190676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.190977 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.191080 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 22:46:36.191190 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 22:46:36.191403 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":249}
I20260809 22:46:36.192420 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.184241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:38961 (local address 127.24.136.1:38659)
0809 22:46:36.184484 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.184491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.184514 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:46:36.184617 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.184622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.184701 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.184802 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:36.185237 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.186426 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.189168 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.190433 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.190621 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.190692 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.191222 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.191368 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:46:36.193216 5361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7fbb6764abd446f9c0bd49216ccf86a" candidate_uuid: "71a825ab55274756b03f4c319c77286a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88bc57d6506741ff97979bf46d4b9959" is_pre_election: true
I20260809 22:46:36.193349 5137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7fbb6764abd446f9c0bd49216ccf86a" candidate_uuid: "71a825ab55274756b03f4c319c77286a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b0df5f2b5834104a810228d4ccd1052" is_pre_election: true
I20260809 22:46:36.193850 5361 raft_consensus.cc:2454] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71a825ab55274756b03f4c319c77286a in term 0.
I20260809 22:46:36.193850 5137 raft_consensus.cc:2454] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71a825ab55274756b03f4c319c77286a in term 0.
I20260809 22:46:36.194561 5478 leader_election.cc:302] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b0df5f2b5834104a810228d4ccd1052, 71a825ab55274756b03f4c319c77286a; no voters:
I20260809 22:46:36.194958 5718 raft_consensus.cc:2781] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:36.195144 5718 raft_consensus.cc:482] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:36.195266 5718 raft_consensus.cc:3037] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:36.197576 5718 raft_consensus.cc:504] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:36.198231 5718 leader_election.cc:288] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [CANDIDATE]: Term 1 election: Requested vote from peers 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1:38659), 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875)
I20260809 22:46:36.198764 5137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7fbb6764abd446f9c0bd49216ccf86a" candidate_uuid: "71a825ab55274756b03f4c319c77286a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b0df5f2b5834104a810228d4ccd1052"
I20260809 22:46:36.198827 5361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7fbb6764abd446f9c0bd49216ccf86a" candidate_uuid: "71a825ab55274756b03f4c319c77286a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88bc57d6506741ff97979bf46d4b9959"
I20260809 22:46:36.198953 5137 raft_consensus.cc:3037] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:36.199055 5361 raft_consensus.cc:3037] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:36.201658 5361 raft_consensus.cc:2454] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71a825ab55274756b03f4c319c77286a in term 1.
I20260809 22:46:36.201767 5137 raft_consensus.cc:2454] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71a825ab55274756b03f4c319c77286a in term 1.
I20260809 22:46:36.202087 5478 leader_election.cc:302] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71a825ab55274756b03f4c319c77286a, 88bc57d6506741ff97979bf46d4b9959; no voters:
I20260809 22:46:36.202446 5718 raft_consensus.cc:2781] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:36.202909 5718 raft_consensus.cc:686] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 1 LEADER]: Becoming Leader. State: Replica: 71a825ab55274756b03f4c319c77286a, State: Running, Role: LEADER
I20260809 22:46:36.203725 5718 consensus_queue.cc:237] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:36.209128 3600 catalog_manager.cc:5495] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a reported cstate change: term changed from 0 to 1, leader changed from <none> to 71a825ab55274756b03f4c319c77286a (127.24.136.3). New cstate: current_term: 1 leader_uuid: "71a825ab55274756b03f4c319c77286a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } health_report { overall_health: UNKNOWN } } }
W20260809 22:46:36.230615 5657 tablet.cc:2369] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:36.245524 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.239855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.24.136.3:44141)
0809 22:46:36.240071 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.240078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.240167 (+ 89us) server_negotiation.cc:407] Connection header received
0809 22:46:36.240391 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.240395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.240492 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.240629 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:36.240929 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.241654 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.243005 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.243529 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.243654 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.244296 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:36.244868 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:36.244994 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.245165 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.245247 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:46:36.254470 5361 raft_consensus.cc:1261] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [term 1 FOLLOWER]: Refusing update from remote peer 71a825ab55274756b03f4c319c77286a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:36.254475 5137 raft_consensus.cc:1261] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [term 1 FOLLOWER]: Refusing update from remote peer 71a825ab55274756b03f4c319c77286a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:36.255563 5722 consensus_queue.cc:1035] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [LEADER]: Connected to new peer: Peer: permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:36.255972 5718 consensus_queue.cc:1035] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:36.285367 3603 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:36.285739 3603 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:36.296977 3603 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b23e1a1756f4ebabb8c7d61ce3b1f9d: Sending DeleteTablet for 3 replicas of tablet e7fbb6764abd446f9c0bd49216ccf86a
I20260809 22:46:36.298249 5552 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fbb6764abd446f9c0bd49216ccf86a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:36 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:35436
I20260809 22:46:36.299196 5550 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fbb6764abd446f9c0bd49216ccf86a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:36 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:35436
I20260809 22:46:36.299975 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fbb6764abd446f9c0bd49216ccf86a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:36 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:60042
I20260809 22:46:36.300036 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fbb6764abd446f9c0bd49216ccf86a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:36 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:36300
I20260809 22:46:36.299940 5094 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fbb6764abd446f9c0bd49216ccf86a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:36 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:36300
I20260809 22:46:36.325649 5748 tablet_replica.cc:330] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: stopping tablet replica
I20260809 22:46:36.326133 5748 raft_consensus.cc:2229] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:36.326634 5748 raft_consensus.cc:2258] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:36.328262 5746 tablet_replica.cc:330] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: stopping tablet replica
I20260809 22:46:36.328723 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 3538
I20260809 22:46:36.328778 5746 raft_consensus.cc:2229] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:36.329411 5746 raft_consensus.cc:2258] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:36.330312 3650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:36.332772 5745 tablet_replica.cc:330] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: stopping tablet replica
I20260809 22:46:36.333313 5745 raft_consensus.cc:2229] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:36.334053 5745 raft_consensus.cc:2258] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:36.336133 5746 ts_tablet_manager.cc:1905] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:36.335379 5748 ts_tablet_manager.cc:1905] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:36.347389 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.301202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.24.136.61:37453)
0809 22:46:36.301386 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.301392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.327653 (+ 26261us) server_negotiation.cc:407] Connection header received
0809 22:46:36.328391 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.328398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.328494 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.328620 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:36.331559 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.332696 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.334835 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.335524 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.335657 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.340147 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.340155 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.340160 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.340209 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.343821 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.344894 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.344901 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.344905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.345073 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.345551 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.345556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.345559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.345975 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.346198 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.346787 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.347029 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:46:36.358387 5745 ts_tablet_manager.cc:1905] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:36.362872 5748 ts_tablet_manager.cc:1918] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:36.363090 5748 log.cc:1196] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/e7fbb6764abd446f9c0bd49216ccf86a
I20260809 22:46:36.363761 5748 ts_tablet_manager.cc:1939] T e7fbb6764abd446f9c0bd49216ccf86a P 1b0df5f2b5834104a810228d4ccd1052: Deleting consensus metadata
I20260809 22:46:36.366866 5746 ts_tablet_manager.cc:1918] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:36.367061 5746 log.cc:1196] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/e7fbb6764abd446f9c0bd49216ccf86a
I20260809 22:46:36.367180 5745 ts_tablet_manager.cc:1918] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:36.367386 5745 log.cc:1196] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/e7fbb6764abd446f9c0bd49216ccf86a
I20260809 22:46:36.367714 5746 ts_tablet_manager.cc:1939] T e7fbb6764abd446f9c0bd49216ccf86a P 88bc57d6506741ff97979bf46d4b9959: Deleting consensus metadata
I20260809 22:46:36.368016 5745 ts_tablet_manager.cc:1939] T e7fbb6764abd446f9c0bd49216ccf86a P 71a825ab55274756b03f4c319c77286a: Deleting consensus metadata
I20260809 22:46:36.375872 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.328012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60440 (local address 127.24.136.60:42669)
0809 22:46:36.328230 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.328236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.350071 (+ 21835us) server_negotiation.cc:407] Connection header received
0809 22:46:36.350382 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.350388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.350490 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.350556 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:36.352565 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.353664 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.355661 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.356312 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.356419 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.367342 (+ 10923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.367351 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.367356 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.367412 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.371681 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.372878 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.372885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.372889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.373072 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.373666 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.373673 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.373677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.374134 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.374406 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.375120 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.375508 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 22:46:36.504644 3566 catalog_manager.cc:4841] TS 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1:38659): tablet e7fbb6764abd446f9c0bd49216ccf86a (table test-table [id=94afbf7f9f5e4e22bfc91e68c08022ed]) successfully deleted
I20260809 22:46:36.504623 3572 catalog_manager.cc:4841] TS 71a825ab55274756b03f4c319c77286a (127.24.136.3:44141): tablet e7fbb6764abd446f9c0bd49216ccf86a (table test-table [id=94afbf7f9f5e4e22bfc91e68c08022ed]) successfully deleted
I20260809 22:46:36.505287 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 3538
W20260809 22:46:36.505215 3572 catalog_manager.cc:4808] TS 71a825ab55274756b03f4c319c77286a (127.24.136.3:44141): delete failed for tablet e7fbb6764abd446f9c0bd49216ccf86a because the tablet was not found. No further retry: Not found: Tablet not found: e7fbb6764abd446f9c0bd49216ccf86a
W20260809 22:46:36.534597 5255 connection.cc:535] server connection from 127.0.0.1:60042 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 22:46:36.558288 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.539308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60452 (local address 127.24.136.60:42669)
0809 22:46:36.539643 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.539650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.540570 (+ 920us) server_negotiation.cc:407] Connection header received
0809 22:46:36.540898 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.540904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.540994 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.541071 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:36.543569 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.544702 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.546725 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.547447 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.547580 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.551773 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.551781 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.551786 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.551827 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.554945 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.555791 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.555796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.555800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.555978 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.556495 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.556501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.556504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.556916 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.557269 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.557564 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.557834 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:46:36.564749 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.540652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46138 (local address 127.24.136.61:37453)
0809 22:46:36.540860 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.540867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.549697 (+ 8830us) server_negotiation.cc:407] Connection header received
0809 22:46:36.549854 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.549860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.549953 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.550047 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:36.551794 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.552861 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.554651 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.555334 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.555514 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.558672 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.558679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.558683 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.558733 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.561740 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.562460 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.562467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.562471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.562630 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.563014 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.563019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.563021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.563347 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.563654 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.564159 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.564414 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:46:37.280042 5769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.279075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.2:58291)
0809 22:46:37.279669 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.279835 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":277,"threads_started":1}
W20260809 22:46:37.280596 5433 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:42691 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:37.299763 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.298812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.3:47519)
0809 22:46:37.299337 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.299543 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":271,"threads_started":1}
W20260809 22:46:37.300211 5651 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:42691 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:37.302057 5775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.301314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:33531)
0809 22:46:37.301772 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.301897 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":211,"threads_started":1}
W20260809 22:46:37.302501 5204 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:42691 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:37.796439 5777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b23e1a1756f4ebabb8c7d61ce3b1f9d)
I20260809 22:46:37.796865 5777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } }
I20260809 22:46:37.797080 5778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b23e1a1756f4ebabb8c7d61ce3b1f9d)
I20260809 22:46:37.797379 5778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } }
I20260809 22:46:37.798416 5777 leader_election.cc:288] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691), 1e63d468dfbe4118851b84cb8dbefe4a (127.24.136.61:37453)
I20260809 22:46:37.798902 5778 leader_election.cc:288] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691), 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60:42669)
I20260809 22:46:37.800117 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.798837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44564)
0809 22:46:37.799589 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.799839 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":425,"threads_started":1}
W20260809 22:46:37.801555 4876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:37.801915 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.799156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44568)
0809 22:46:37.801331 (+ 2175us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.801581 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2029,"mutex_wait_us":144,"thread_start_us":1654,"threads_started":1}
W20260809 22:46:37.802363 4559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:37.808934 5781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.799156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:42669 (local address 127.0.0.1:60468)
0809 22:46:37.799917 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.799935 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:37.800093 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:37.801872 (+ 1779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:37.801877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:37.801903 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:37.802285 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:37.802300 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.804243 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.804248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:37.806348 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:37.806362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.808078 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.808084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:37.808296 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:37.808330 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:46:37.808404 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 22:46:37.808560 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":1038,"threads_started":1}
I20260809 22:46:37.808979 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.799705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:37453 (local address 127.0.0.1:46154)
0809 22:46:37.800485 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.800509 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:37.800650 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:37.801818 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:37.801823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:37.801866 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:37.802129 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:37.802143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.804189 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.804194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:37.806343 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:37.806360 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.808242 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.808248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:37.808433 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:37.808467 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:46:37.808567 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 22:46:37.808715 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":810,"threads_started":1}
I20260809 22:46:37.809662 5784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.799811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.24.136.61:37453)
0809 22:46:37.801209 (+ 1398us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:37.801215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:37.801237 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:46:37.801314 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:37.801319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:37.801414 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:37.801536 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:37.802643 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.804018 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.806541 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.807867 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.807998 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:37.808124 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:37.809183 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0809 22:46:37.809328 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1246,"thread_start_us":1039,"threads_started":1}
I20260809 22:46:37.810565 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.800753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.24.136.60:42669)
0809 22:46:37.801547 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:37.801553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:37.801571 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:46:37.801632 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:37.801641 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:37.801715 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:37.802454 (+ 739us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:37.802751 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.804085 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.806551 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.807879 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.809688 (+ 1809us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:37.809974 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:37.810029 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 22:46:37.810169 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":538,"threads_started":1}
I20260809 22:46:37.812247 5781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.811807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44578)
0809 22:46:37.812012 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.812122 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260809 22:46:37.812438 4559 leader_election.cc:334] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691): Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:37.812829 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.812382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44584)
0809 22:46:37.812559 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.812673 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 22:46:37.812983 4926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "06d41c5d02d542d9a6b1537b109a31ee" is_pre_election: true
W20260809 22:46:37.813045 4876 leader_election.cc:334] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691): Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:37.813252 4598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06d41c5d02d542d9a6b1537b109a31ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" is_pre_election: true
I20260809 22:46:37.813366 4926 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63d468dfbe4118851b84cb8dbefe4a in term 1.
I20260809 22:46:37.813584 4598 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d41c5d02d542d9a6b1537b109a31ee in term 1.
I20260809 22:46:37.813875 4561 leader_election.cc:302] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d41c5d02d542d9a6b1537b109a31ee, 1e63d468dfbe4118851b84cb8dbefe4a; no voters: 8b23e1a1756f4ebabb8c7d61ce3b1f9d
I20260809 22:46:37.814113 4876 leader_election.cc:302] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d41c5d02d542d9a6b1537b109a31ee, 1e63d468dfbe4118851b84cb8dbefe4a; no voters: 8b23e1a1756f4ebabb8c7d61ce3b1f9d
I20260809 22:46:37.814292 5778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 22:46:37.814424 5778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b23e1a1756f4ebabb8c7d61ce3b1f9d)
I20260809 22:46:37.814519 5778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:46:37.814532 5777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 22:46:37.814662 5777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b23e1a1756f4ebabb8c7d61ce3b1f9d)
I20260809 22:46:37.814740 5777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:46:37.816862 5777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } }
I20260809 22:46:37.816862 5778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } }
I20260809 22:46:37.817507 5778 leader_election.cc:288] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 2 election: Requested vote from peers 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691), 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60:42669)
I20260809 22:46:37.817546 5777 leader_election.cc:288] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [CANDIDATE]: Term 2 election: Requested vote from peers 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691), 1e63d468dfbe4118851b84cb8dbefe4a (127.24.136.61:37453)
I20260809 22:46:37.818073 5781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.817649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44596)
0809 22:46:37.817810 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.817900 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 22:46:37.818181 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.817804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44610)
0809 22:46:37.817953 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.818035 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 22:46:37.818435 4598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06d41c5d02d542d9a6b1537b109a31ee" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1e63d468dfbe4118851b84cb8dbefe4a"
I20260809 22:46:37.818722 4598 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 06d41c5d02d542d9a6b1537b109a31ee in current term 2: Already voted for candidate 1e63d468dfbe4118851b84cb8dbefe4a in this term.
I20260809 22:46:37.819193 5781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.818821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44622)
0809 22:46:37.818974 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.819044 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 22:46:37.819352 4559 leader_election.cc:334] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691): Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:37.819622 4926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "06d41c5d02d542d9a6b1537b109a31ee"
I20260809 22:46:37.819890 4926 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1e63d468dfbe4118851b84cb8dbefe4a in current term 2: Already voted for candidate 06d41c5d02d542d9a6b1537b109a31ee in this term.
I20260809 22:46:37.820138 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.819722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44624)
0809 22:46:37.819902 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.819987 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 22:46:37.820273 4876 leader_election.cc:334] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691): Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:37.820338 4561 leader_election.cc:302] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e63d468dfbe4118851b84cb8dbefe4a; no voters: 06d41c5d02d542d9a6b1537b109a31ee, 8b23e1a1756f4ebabb8c7d61ce3b1f9d
I20260809 22:46:37.820431 4876 leader_election.cc:302] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06d41c5d02d542d9a6b1537b109a31ee; no voters: 1e63d468dfbe4118851b84cb8dbefe4a, 8b23e1a1756f4ebabb8c7d61ce3b1f9d
I20260809 22:46:37.820624 5778 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 22:46:37.820703 5777 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 22:46:38.282359 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:38.281538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.2:39385)
0809 22:46:38.282056 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:38.282168 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":291,"threads_started":1}
I20260809 22:46:38.302043 5798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:38.301147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.3:44405)
0809 22:46:38.301671 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:38.301771 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":274,"threads_started":1}
I20260809 22:46:38.304003 5799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:38.303193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:49655)
0809 22:46:38.303744 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:38.303829 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":314,"threads_started":1}
I20260809 22:46:39.284647 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.283505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.2:57707)
0809 22:46:39.284252 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.284382 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":462,"threads_started":1}
I20260809 22:46:39.304153 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.303286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.3:45705)
0809 22:46:39.303870 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.303974 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":308,"threads_started":1}
I20260809 22:46:39.305668 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.304886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:41813)
0809 22:46:39.305378 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.305483 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":250,"threads_started":1}
I20260809 22:46:39.537900 5805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:39.538125 5805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } }
I20260809 22:46:39.538823 5805 leader_election.cc:288] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691), 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60:42669)
I20260809 22:46:39.539533 4926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "06d41c5d02d542d9a6b1537b109a31ee" is_pre_election: true
I20260809 22:46:39.539832 4926 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63d468dfbe4118851b84cb8dbefe4a in term 2.
I20260809 22:46:39.539954 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.539019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44852)
0809 22:46:39.539573 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.539699 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":239,"threads_started":1}
I20260809 22:46:39.540333 4561 leader_election.cc:302] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d41c5d02d542d9a6b1537b109a31ee, 1e63d468dfbe4118851b84cb8dbefe4a; no voters:
I20260809 22:46:39.540658 5805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 22:46:39.540773 5805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:39.540876 5805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:46:39.541429 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.541033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44854)
0809 22:46:39.541188 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.541286 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:46:39.541651 4559 leader_election.cc:334] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691): Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:39.543213 5805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } }
I20260809 22:46:39.543828 5805 leader_election.cc:288] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 3 election: Requested vote from peers 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691), 06d41c5d02d542d9a6b1537b109a31ee (127.24.136.60:42669)
I20260809 22:46:39.544396 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.544003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44860)
0809 22:46:39.544170 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.544260 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 22:46:39.544376 4926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "06d41c5d02d542d9a6b1537b109a31ee"
I20260809 22:46:39.544641 4926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:46:39.545279 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.544985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44862)
0809 22:46:39.545108 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.545173 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:46:39.545442 4559 leader_election.cc:334] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691): Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
I20260809 22:46:39.546984 4926 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63d468dfbe4118851b84cb8dbefe4a in term 3.
I20260809 22:46:39.547449 4561 leader_election.cc:302] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d41c5d02d542d9a6b1537b109a31ee, 1e63d468dfbe4118851b84cb8dbefe4a; no voters: 8b23e1a1756f4ebabb8c7d61ce3b1f9d
I20260809 22:46:39.547802 5805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 22:46:39.548213 5805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [term 3 LEADER]: Becoming Leader. State: Replica: 1e63d468dfbe4118851b84cb8dbefe4a, State: Running, Role: LEADER
I20260809 22:46:39.548601 5805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } }
I20260809 22:46:39.550558 5811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e63d468dfbe4118851b84cb8dbefe4a. Latest consensus state: current_term: 3 leader_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:39.550920 5811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:39.551597 5814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:39.555881 5814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:39.556826 5814 catalog_manager.cc:1259] Loaded cluster ID: dd6b1a6cdddb4375a36414e5c98b8041
I20260809 22:46:39.556910 5814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:39.558120 5814 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:39.559020 5814 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Loaded TSK: 0
I20260809 22:46:39.560025 5814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:46:39.618638 4579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:46:39.625653 4579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:39.630471 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.629950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44878)
0809 22:46:39.630179 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.630302 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 22:46:39.630553 4926 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [term 3 FOLLOWER]: Refusing update from remote peer 1e63d468dfbe4118851b84cb8dbefe4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 22:46:39.631421 5811 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 22:46:39.631762 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.631235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44886)
0809 22:46:39.631430 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.631561 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 22:46:39.631914 4559 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a -> Peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d (127.24.136.62:42691): Couldn't send request to peer 8b23e1a1756f4ebabb8c7d61ce3b1f9d. Status: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 22:46:39.634594 5818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e63d468dfbe4118851b84cb8dbefe4a. Latest consensus state: current_term: 3 leader_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:39.634927 5818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:39.636288 5811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:39.636624 5811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:39.636667 5818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "8b23e1a1756f4ebabb8c7d61ce3b1f9d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42691 } } peers { permanent_uuid: "1e63d468dfbe4118851b84cb8dbefe4a" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 37453 } attrs { promote: false } } peers { permanent_uuid: "06d41c5d02d542d9a6b1537b109a31ee" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 42669 } attrs { promote: false } } }
I20260809 22:46:39.637219 5818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06d41c5d02d542d9a6b1537b109a31ee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:39.655391 5833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.646291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:38659 (local address 127.0.0.1:36308)
0809 22:46:39.646877 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.647186 (+ 309us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:39.647335 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:39.648022 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:39.648028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:39.648068 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:39.648506 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.648519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.650191 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.650197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.652495 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.652517 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.654373 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.654380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.654598 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.654639 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:46:39.654744 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 22:46:39.654909 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":550,"threads_started":1}
I20260809 22:46:39.656021 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.646692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.24.136.1:38659)
0809 22:46:39.647154 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:39.647160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:39.647257 (+ 97us) server_negotiation.cc:407] Connection header received
0809 22:46:39.647559 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:39.647567 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:39.647672 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:39.647750 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:39.648730 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.649951 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.652743 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.654159 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.654321 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.654450 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:39.654779 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 22:46:39.654935 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":1144,"threads_started":1}
I20260809 22:46:39.657027 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 15146d89ddd444e2b5fe19642f5d00aa (DEFAULT_TABLE table=test-table [id=a703773b7e5e4a4c8b78460ac46fce8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:39.657584 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15146d89ddd444e2b5fe19642f5d00aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:39.665314 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.647651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.24.136.2:45875)
0809 22:46:39.659401 (+ 11750us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:39.659406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:39.659423 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:46:39.659520 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:39.659526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:39.659615 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:39.660551 (+ 936us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:39.660774 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.661661 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.663419 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.664285 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.664403 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.664510 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:39.664831 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 22:46:39.664971 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11620,"thread_start_us":12064,"threads_started":1}
I20260809 22:46:39.666270 5837 tablet_bootstrap.cc:492] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: Bootstrap starting.
I20260809 22:46:39.666720 5835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.647187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:45875 (local address 127.0.0.1:60048)
0809 22:46:39.647721 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.647743 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:39.647865 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:39.659919 (+ 12054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:39.659925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:39.659959 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:39.660376 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.660390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.661815 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.661820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.663264 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.663277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.664502 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.664509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.664667 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.664692 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:46:39.664782 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 22:46:39.664925 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":3,"thread_start_us":898,"threads_started":1}
I20260809 22:46:39.667845 5334 tablet_service.cc:1467] Processing CreateTablet for tablet 15146d89ddd444e2b5fe19642f5d00aa (DEFAULT_TABLE table=test-table [id=a703773b7e5e4a4c8b78460ac46fce8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:39.668156 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.645888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44141 (local address 127.0.0.1:35452)
0809 22:46:39.646084 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.646105 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:39.646257 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:39.661871 (+ 15614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:39.661875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:39.661907 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:39.662275 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.662285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.663728 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.663734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.665743 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.665762 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.667399 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.667406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.667595 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.667621 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:46:39.667711 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 22:46:39.667860 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":22}
I20260809 22:46:39.668217 5830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.646025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.24.136.3:44141)
0809 22:46:39.661530 (+ 15505us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:39.661536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:39.661568 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:46:39.661640 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:39.661645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:39.661723 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:39.661846 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:39.662442 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.663558 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.665978 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.667180 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.667296 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.667407 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:39.667765 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 22:46:39.667908 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15359,"thread_start_us":15953,"threads_started":1}
I20260809 22:46:39.668226 5837 tablet_bootstrap.cc:654] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:39.668308 5334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15146d89ddd444e2b5fe19642f5d00aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:39.668874 5552 tablet_service.cc:1467] Processing CreateTablet for tablet 15146d89ddd444e2b5fe19642f5d00aa (DEFAULT_TABLE table=test-table [id=a703773b7e5e4a4c8b78460ac46fce8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:39.669247 5552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15146d89ddd444e2b5fe19642f5d00aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:39.670608 5837 tablet_bootstrap.cc:492] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: No bootstrap required, opened a new log
I20260809 22:46:39.670744 5837 ts_tablet_manager.cc:1397] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 22:46:39.671640 5837 raft_consensus.cc:348] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:39.671917 5837 raft_consensus.cc:374] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:39.672036 5837 raft_consensus.cc:729] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0df5f2b5834104a810228d4ccd1052, State: Initialized, Role: FOLLOWER
I20260809 22:46:39.672356 5837 consensus_queue.cc:260] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:39.673321 5837 ts_tablet_manager.cc:1428] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 22:46:39.674134 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.673638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:54925)
0809 22:46:39.673796 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.673915 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:46:39.675165 5840 tablet_bootstrap.cc:492] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: Bootstrap starting.
I20260809 22:46:39.677127 5840 tablet_bootstrap.cc:654] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:39.678078 5208 heartbeater.cc:503] Master 127.24.136.61:37453 was elected leader, sending a full tablet report...
I20260809 22:46:39.678078 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.677646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:38973)
0809 22:46:39.677805 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.677895 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 22:46:39.679649 5840 tablet_bootstrap.cc:492] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: No bootstrap required, opened a new log
I20260809 22:46:39.679881 5840 ts_tablet_manager.cc:1397] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 22:46:39.680686 5840 raft_consensus.cc:348] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:39.680895 5840 raft_consensus.cc:374] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:39.681010 5840 raft_consensus.cc:729] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88bc57d6506741ff97979bf46d4b9959, State: Initialized, Role: FOLLOWER
I20260809 22:46:39.681259 5840 consensus_queue.cc:260] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:39.682044 5840 ts_tablet_manager.cc:1428] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:46:39.683194 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.682741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.2:37363)
0809 22:46:39.682915 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.683027 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:46:39.684401 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.683950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.2:49331)
0809 22:46:39.684153 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.684236 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 22:46:39.684664 5441 heartbeater.cc:503] Master 127.24.136.61:37453 was elected leader, sending a full tablet report...
I20260809 22:46:39.688470 5844 tablet_bootstrap.cc:492] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: Bootstrap starting.
I20260809 22:46:39.690351 5844 tablet_bootstrap.cc:654] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:39.692580 5844 tablet_bootstrap.cc:492] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: No bootstrap required, opened a new log
I20260809 22:46:39.692713 5844 ts_tablet_manager.cc:1397] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 22:46:39.693334 5844 raft_consensus.cc:348] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:39.693527 5844 raft_consensus.cc:374] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:39.693602 5844 raft_consensus.cc:729] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71a825ab55274756b03f4c319c77286a, State: Initialized, Role: FOLLOWER
I20260809 22:46:39.693814 5844 consensus_queue.cc:260] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:39.694989 5844 ts_tablet_manager.cc:1428] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:46:39.695331 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.694884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.3:57221)
0809 22:46:39.695077 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.695178 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 22:46:39.696403 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.695969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.3:56863)
0809 22:46:39.696146 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.696241 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 22:46:39.697063 5656 heartbeater.cc:503] Master 127.24.136.61:37453 was elected leader, sending a full tablet report...
I20260809 22:46:39.801653 5839 raft_consensus.cc:482] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:39.801898 5839 raft_consensus.cc:504] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:39.803531 5839 leader_election.cc:288] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71a825ab55274756b03f4c319c77286a (127.24.136.3:44141), 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875)
I20260809 22:46:39.810184 5830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.804274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:46701 (local address 127.24.136.3:44141)
0809 22:46:39.804495 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:39.804501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:39.804519 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:46:39.804672 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:39.804677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:39.804786 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:39.804934 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:39.805878 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.806610 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.808463 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.809265 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.809427 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.809506 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:39.809805 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 22:46:39.809938 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:46:39.810220 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.803818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44141 (local address 127.24.136.1:46701)
0809 22:46:39.804083 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.804103 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:39.804252 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:39.805323 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:39.805329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:39.805359 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:39.805711 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.805729 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.806751 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.806756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.808299 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.808312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.809463 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.809470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.809634 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.809663 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 22:46:39.809754 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:46:39.809900 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":3}
I20260809 22:46:39.810367 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.804418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:41833 (local address 127.24.136.2:45875)
0809 22:46:39.805137 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:39.805143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:39.805163 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:39.805237 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:39.805242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:39.805320 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:39.805408 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:39.806097 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.806855 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.808656 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.809427 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.809580 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.809657 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:39.809959 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 22:46:39.810098 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539}
I20260809 22:46:39.811055 5583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15146d89ddd444e2b5fe19642f5d00aa" candidate_uuid: "1b0df5f2b5834104a810228d4ccd1052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71a825ab55274756b03f4c319c77286a" is_pre_election: true
I20260809 22:46:39.810669 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.803854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:45875 (local address 127.24.136.1:41833)
0809 22:46:39.804646 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.804665 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:39.804806 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:39.805554 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:39.805559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:39.805580 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:39.805948 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.805957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.806989 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.806993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.808493 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.808506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.809667 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.809674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.809811 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.809836 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:46:39.809915 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 22:46:39.810048 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":36,"thread_start_us":1359,"threads_started":1}
I20260809 22:46:39.811498 5583 raft_consensus.cc:2454] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b0df5f2b5834104a810228d4ccd1052 in term 0.
I20260809 22:46:39.812067 5361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15146d89ddd444e2b5fe19642f5d00aa" candidate_uuid: "1b0df5f2b5834104a810228d4ccd1052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88bc57d6506741ff97979bf46d4b9959" is_pre_election: true
I20260809 22:46:39.812134 5027 leader_election.cc:302] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b0df5f2b5834104a810228d4ccd1052, 71a825ab55274756b03f4c319c77286a; no voters:
I20260809 22:46:39.812340 5361 raft_consensus.cc:2454] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b0df5f2b5834104a810228d4ccd1052 in term 0.
I20260809 22:46:39.812582 5839 raft_consensus.cc:2781] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:39.812728 5839 raft_consensus.cc:482] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:39.812855 5839 raft_consensus.cc:3037] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:39.815099 5839 raft_consensus.cc:504] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:39.815743 5839 leader_election.cc:288] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [CANDIDATE]: Term 1 election: Requested vote from peers 71a825ab55274756b03f4c319c77286a (127.24.136.3:44141), 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875)
I20260809 22:46:39.816256 5583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15146d89ddd444e2b5fe19642f5d00aa" candidate_uuid: "1b0df5f2b5834104a810228d4ccd1052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71a825ab55274756b03f4c319c77286a"
I20260809 22:46:39.816265 5361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15146d89ddd444e2b5fe19642f5d00aa" candidate_uuid: "1b0df5f2b5834104a810228d4ccd1052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88bc57d6506741ff97979bf46d4b9959"
I20260809 22:46:39.816448 5583 raft_consensus.cc:3037] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:39.816450 5361 raft_consensus.cc:3037] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:39.818681 5361 raft_consensus.cc:2454] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b0df5f2b5834104a810228d4ccd1052 in term 1.
I20260809 22:46:39.818711 5583 raft_consensus.cc:2454] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b0df5f2b5834104a810228d4ccd1052 in term 1.
I20260809 22:46:39.819224 5027 leader_election.cc:302] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b0df5f2b5834104a810228d4ccd1052, 71a825ab55274756b03f4c319c77286a; no voters:
I20260809 22:46:39.819610 5839 raft_consensus.cc:2781] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:39.820050 5839 raft_consensus.cc:686] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 1 LEADER]: Becoming Leader. State: Replica: 1b0df5f2b5834104a810228d4ccd1052, State: Running, Role: LEADER
I20260809 22:46:39.820472 5839 consensus_queue.cc:237] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } }
I20260809 22:46:39.823696 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.823237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:42483)
0809 22:46:39.823409 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.823540 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 22:46:39.824527 4582 catalog_manager.cc:5495] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "1b0df5f2b5834104a810228d4ccd1052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b0df5f2b5834104a810228d4ccd1052" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 } health_report { overall_health: UNKNOWN } } }
I20260809 22:46:39.825261 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.824891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:33037)
0809 22:46:39.825047 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.825121 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:46:39.847127 5443 tablet.cc:2369] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:39.854192 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.849295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.24.136.1:38659)
0809 22:46:39.849494 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:39.849499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:39.849518 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:46:39.849626 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:39.849630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:39.849722 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:39.849854 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:39.850138 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.850878 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.852191 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.852649 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.852743 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.853074 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:39.853628 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:39.853736 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:39.853841 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 22:46:39.853914 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:46:39.858695 5361 raft_consensus.cc:1261] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [term 1 FOLLOWER]: Refusing update from remote peer 1b0df5f2b5834104a810228d4ccd1052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:39.859074 5583 raft_consensus.cc:1261] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [term 1 FOLLOWER]: Refusing update from remote peer 1b0df5f2b5834104a810228d4ccd1052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:39.859512 5839 consensus_queue.cc:1035] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88bc57d6506741ff97979bf46d4b9959" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:39.859891 5854 consensus_queue.cc:1035] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71a825ab55274756b03f4c319c77286a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:39.863113 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.862536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.2:49993)
0809 22:46:39.862810 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.862919 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260809 22:46:39.864333 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.863784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.3:34703)
0809 22:46:39.864002 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.864128 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:46:39.868942 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.868405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.3:60411)
0809 22:46:39.868636 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.868744 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260809 22:46:39.869720 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.869258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:33287)
0809 22:46:39.869432 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.869529 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:46:39.872736 5861 mvcc.cc:205] Tried to move back new op lower bound from 7316748697013534720 to 7316748696868626432. Current Snapshot: MvccSnapshot[applied={T|T < 7316748697013534720 or (T in {7316748697013534720})}]
I20260809 22:46:39.884506 4582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:39.884837 4582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:39.885735 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.884948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.1:33431)
0809 22:46:39.885417 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.885535 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321}
I20260809 22:46:39.888489 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.887815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.24.136.2:41705)
0809 22:46:39.888170 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.888285 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260809 22:46:39.894227 4582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e63d468dfbe4118851b84cb8dbefe4a: Sending DeleteTablet for 3 replicas of tablet 15146d89ddd444e2b5fe19642f5d00aa
I20260809 22:46:39.895682 5094 tablet_service.cc:1514] Processing DeleteTablet for tablet 15146d89ddd444e2b5fe19642f5d00aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:39 UTC) from {username='oryx', principal='oryx/127.24.136.61@KRBTEST.COM'} at 127.0.0.1:36308
I20260809 22:46:39.895684 5552 tablet_service.cc:1514] Processing DeleteTablet for tablet 15146d89ddd444e2b5fe19642f5d00aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:39 UTC) from {username='oryx', principal='oryx/127.24.136.61@KRBTEST.COM'} at 127.0.0.1:35452
I20260809 22:46:39.895685 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 15146d89ddd444e2b5fe19642f5d00aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:39 UTC) from {username='oryx', principal='oryx/127.24.136.61@KRBTEST.COM'} at 127.0.0.1:36308
I20260809 22:46:39.896595 5334 tablet_service.cc:1514] Processing DeleteTablet for tablet 15146d89ddd444e2b5fe19642f5d00aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:39 UTC) from {username='oryx', principal='oryx/127.24.136.61@KRBTEST.COM'} at 127.0.0.1:60048
I20260809 22:46:39.896713 5333 tablet_service.cc:1514] Processing DeleteTablet for tablet 15146d89ddd444e2b5fe19642f5d00aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:39 UTC) from {username='oryx', principal='oryx/127.24.136.61@KRBTEST.COM'} at 127.0.0.1:60048
I20260809 22:46:39.897353 5881 tablet_replica.cc:330] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: stopping tablet replica
I20260809 22:46:39.897620 5881 raft_consensus.cc:2229] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:39.897969 5881 raft_consensus.cc:2258] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:39.898294 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 4984
I20260809 22:46:39.899298 5881 ts_tablet_manager.cc:1905] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:39.906737 5881 ts_tablet_manager.cc:1918] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:39.907003 5881 log.cc:1196] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/15146d89ddd444e2b5fe19642f5d00aa
I20260809 22:46:39.907774 5881 ts_tablet_manager.cc:1939] T 15146d89ddd444e2b5fe19642f5d00aa P 88bc57d6506741ff97979bf46d4b9959: Deleting consensus metadata
I20260809 22:46:39.909435 4559 catalog_manager.cc:4841] TS 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875): tablet 15146d89ddd444e2b5fe19642f5d00aa (table test-table [id=a703773b7e5e4a4c8b78460ac46fce8b]) successfully deleted
W20260809 22:46:39.910243 4559 catalog_manager.cc:4808] TS 88bc57d6506741ff97979bf46d4b9959 (127.24.136.2:45875): delete failed for tablet 15146d89ddd444e2b5fe19642f5d00aa because the tablet was not found. No further retry: Not found: Tablet not found: 15146d89ddd444e2b5fe19642f5d00aa
I20260809 22:46:39.912377 5880 tablet_replica.cc:330] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: stopping tablet replica
I20260809 22:46:39.912719 5880 raft_consensus.cc:2229] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:39.913532 5880 raft_consensus.cc:2258] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:39.914281 5878 tablet_replica.cc:330] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: stopping tablet replica
I20260809 22:46:39.914501 5878 raft_consensus.cc:2229] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:39.914810 5878 raft_consensus.cc:2258] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:39.916126 5878 ts_tablet_manager.cc:1905] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:39.916803 5880 ts_tablet_manager.cc:1905] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:39.925779 5878 ts_tablet_manager.cc:1918] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:39.926061 5878 log.cc:1196] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/15146d89ddd444e2b5fe19642f5d00aa
I20260809 22:46:39.926601 5878 ts_tablet_manager.cc:1939] T 15146d89ddd444e2b5fe19642f5d00aa P 71a825ab55274756b03f4c319c77286a: Deleting consensus metadata
I20260809 22:46:39.927227 5880 ts_tablet_manager.cc:1918] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:39.927439 5880 log.cc:1196] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/15146d89ddd444e2b5fe19642f5d00aa
I20260809 22:46:39.928062 5880 ts_tablet_manager.cc:1939] T 15146d89ddd444e2b5fe19642f5d00aa P 1b0df5f2b5834104a810228d4ccd1052: Deleting consensus metadata
I20260809 22:46:39.928148 4560 catalog_manager.cc:4841] TS 71a825ab55274756b03f4c319c77286a (127.24.136.3:44141): tablet 15146d89ddd444e2b5fe19642f5d00aa (table test-table [id=a703773b7e5e4a4c8b78460ac46fce8b]) successfully deleted
I20260809 22:46:39.929364 4559 catalog_manager.cc:4841] TS 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1:38659): tablet 15146d89ddd444e2b5fe19642f5d00aa (table test-table [id=a703773b7e5e4a4c8b78460ac46fce8b]) successfully deleted
W20260809 22:46:39.929852 4559 catalog_manager.cc:4808] TS 1b0df5f2b5834104a810228d4ccd1052 (127.24.136.1:38659): delete failed for tablet 15146d89ddd444e2b5fe19642f5d00aa because the tablet was not found. No further retry: Not found: Tablet not found: 15146d89ddd444e2b5fe19642f5d00aa
I20260809 22:46:39.930932 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.899788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36330 (local address 127.24.136.1:38659)
0809 22:46:39.899982 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:39.899988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:39.911497 (+ 11509us) server_negotiation.cc:407] Connection header received
0809 22:46:39.911721 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:39.911727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:39.911844 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:39.911963 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:39.913882 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.915037 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.917024 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.917720 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.917871 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.921598 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:39.921619 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:39.921641 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:39.921693 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:39.926577 (+ 4884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:39.927740 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:39.927751 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:39.927755 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:39.927994 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:39.928580 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:39.928585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:39.928589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:39.929124 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:39.929476 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:39.930246 (+ 770us) server_negotiation.cc:299] Negotiation successful
0809 22:46:39.930525 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:46:39.931749 5197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:40.040858 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 4984
I20260809 22:46:40.067684 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 5230
I20260809 22:46:40.082198 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:40.068573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60056 (local address 127.24.136.2:45875)
0809 22:46:40.068774 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:40.068779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:40.068799 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:40.069020 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:40.069025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:40.069123 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:40.069217 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:40.070612 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:40.071369 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:40.072811 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:40.073306 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:40.073452 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:40.076190 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:40.076210 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:40.076230 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:40.076286 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:40.079080 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:40.079997 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:40.080006 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:40.080008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:40.080211 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:40.080639 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:40.080642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:40.080644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:40.081069 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:40.081342 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:40.081652 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 22:46:40.081845 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:46:40.082954 5424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:40.156078 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:40.155557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44900)
0809 22:46:40.155752 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:40.155874 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 22:46:40.157732 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:40.157319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42691 (local address 127.0.0.1:44906)
0809 22:46:40.157477 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:40.157577 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:42691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:46:40.178668 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 5230
I20260809 22:46:40.212770 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 5457
I20260809 22:46:40.227280 5830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:40.213606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.24.136.3:44141)
0809 22:46:40.213800 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:40.213805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:40.213825 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:40.213935 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:40.213939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:40.214002 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:40.214103 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:40.215617 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:40.216411 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:40.217754 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:40.218217 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:40.218335 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:40.221092 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:40.221110 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:40.221128 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:40.221169 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:40.224278 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:40.225031 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:40.225039 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:40.225041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:40.225180 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:40.225535 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:40.225539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:40.225542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:40.226033 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:40.226280 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:40.226706 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 22:46:40.226931 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:46:40.228039 5644 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:40.349345 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 5457
I20260809 22:46:40.375293 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 4355
I20260809 22:46:40.376482 4634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:40.502439 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 4355
I20260809 22:46:40.533800 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 4526
I20260809 22:46:40.534917 4968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:40.654381 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 4526
2026-08-09T22:46:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11874 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:40 dist-test-slave-pww0 krb5kdc[5894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:40 dist-test-slave-pww0 krb5kdc[5894](info): set up 2 sockets
Aug 09 22:46:40 dist-test-slave-pww0 krb5kdc[5894](info): commencing operation
krb5kdc: starting...
W20260809 22:46:41.922508 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:46:42 dist-test-slave-pww0 krb5kdc[5894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.62.keytab.
Entry for principal oryx/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.62.keytab.
I20260809 22:46:42.196784 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38065
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.62.keytab
--principal=oryx/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:42.547259 5921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:42.547642 5921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:42.547793 5921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:42.555318 5921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:42.555481 5921 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:46:42.555547 5921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:42.555637 5921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:42.555712 5921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:42.555799 5921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:42.566148 5921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38065
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.62.keytab
--principal=oryx/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:42.567622 5921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:42.569036 5921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:42.581357 5929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:42.581357 5927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:42.581796 5932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:42.582370 5921 server_base.cc:1034] running on GCE node
I20260809 22:46:42.583251 5921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:42.584868 5921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:42.586102 5921 hybrid_clock.cc:648] HybridClock initialized: now 1786315602586034 us; error 50 us; skew 500 ppm
I20260809 22:46:42.586457 5921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:42 dist-test-slave-pww0 krb5kdc[5894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315602, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:42.594491 5921 init.cc:377] Logged in from keytab as oryx/127.24.136.62@KRBTEST.COM (short username oryx)
I20260809 22:46:42.596266 5921 webserver.cc:460] Webserver started at http://127.24.136.62:39873/ using document root <none> and password file <none>
I20260809 22:46:42.597046 5921 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:42.597162 5921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:42.597499 5921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:42.600802 5921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "fe2005c200ca4d599ce79202e56f3b58"
format_stamp: "Formatted at 2026-08-09 22:46:42 on dist-test-slave-pww0"
server_key: "8d2c500562e4ba65a7b7aa55ff21b2d2"
server_key_iv: "f76f30c4b92807973d6dfd167a4c1e92"
server_key_version: "encryptionkey@0"
I20260809 22:46:42.601697 5921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "fe2005c200ca4d599ce79202e56f3b58"
format_stamp: "Formatted at 2026-08-09 22:46:42 on dist-test-slave-pww0"
server_key: "8d2c500562e4ba65a7b7aa55ff21b2d2"
server_key_iv: "f76f30c4b92807973d6dfd167a4c1e92"
server_key_version: "encryptionkey@0"
I20260809 22:46:42.607702 5921 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 22:46:42.611573 5945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:42.612990 5921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:42.613170 5921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "fe2005c200ca4d599ce79202e56f3b58"
format_stamp: "Formatted at 2026-08-09 22:46:42 on dist-test-slave-pww0"
server_key: "8d2c500562e4ba65a7b7aa55ff21b2d2"
server_key_iv: "f76f30c4b92807973d6dfd167a4c1e92"
server_key_version: "encryptionkey@0"
I20260809 22:46:42.613394 5921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:42.640125 5921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:42.644837 5921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:42.645117 5921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:42.666858 5921 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38065
I20260809 22:46:42.666865 6036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38065 every 8 connection(s)
I20260809 22:46:42.668279 5921 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:42.672498 6038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:42.674690 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 5921
I20260809 22:46:42.674937 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:42.675817 25120 external_mini_cluster.cc:1428] Setting key a7067a2f48ce904f8d9d807fd50b98f8
I20260809 22:46:42.686007 6038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58: Bootstrap starting.
I20260809 22:46:42.690270 6038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58: Neither blocks nor log segments found. Creating new log.
Aug 09 22:46:42 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:42.691674 6038 log.cc:824] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:42.694733 6038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58: No bootstrap required, opened a new log
I20260809 22:46:42.700383 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:42.678218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47318 (local address 127.24.136.62:38065)
0809 22:46:42.679146 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:42.679160 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:42.679208 (+ 48us) server_negotiation.cc:407] Connection header received
0809 22:46:42.681027 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:42.681056 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:42.681482 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:42.681855 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:42.683824 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:42.685202 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:42.686984 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:42.687792 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:42.687957 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:42.692795 (+ 4838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:42.692805 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:42.692821 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:42.692862 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:42.695953 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:42.696675 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:42.696683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:42.696691 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:42.696891 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:42.697305 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:42.697309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:42.697311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:42.697947 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:42.698270 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:42.698759 (+ 489us) server_negotiation.cc:299] Negotiation successful
0809 22:46:42.699237 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":448,"threads_started":1}
I20260809 22:46:42.702862 6038 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2005c200ca4d599ce79202e56f3b58" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38065 } }
I20260809 22:46:42.703258 6038 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:42.703380 6038 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2005c200ca4d599ce79202e56f3b58, State: Initialized, Role: FOLLOWER
I20260809 22:46:42.703996 6038 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2005c200ca4d599ce79202e56f3b58" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38065 } }
I20260809 22:46:42.704212 6038 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:42.704311 6038 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:42.704438 6038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:42.706591 6038 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2005c200ca4d599ce79202e56f3b58" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38065 } }
I20260809 22:46:42.707113 6038 leader_election.cc:302] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe2005c200ca4d599ce79202e56f3b58; no voters:
I20260809 22:46:42.707722 6038 leader_election.cc:288] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:42.707852 6046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:42.709030 6046 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [term 1 LEADER]: Becoming Leader. State: Replica: fe2005c200ca4d599ce79202e56f3b58, State: Running, Role: LEADER
I20260809 22:46:42.709668 6046 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2005c200ca4d599ce79202e56f3b58" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38065 } }
I20260809 22:46:42.709956 6038 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:42.712783 6051 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe2005c200ca4d599ce79202e56f3b58. Latest consensus state: current_term: 1 leader_uuid: "fe2005c200ca4d599ce79202e56f3b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2005c200ca4d599ce79202e56f3b58" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38065 } } }
I20260809 22:46:42.713050 6048 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe2005c200ca4d599ce79202e56f3b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe2005c200ca4d599ce79202e56f3b58" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38065 } } }
I20260809 22:46:42.713303 6048 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:42.713274 6051 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:42.715197 6061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:42.720753 6061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:42.729591 6061 catalog_manager.cc:1347] Generated new cluster ID: e1bf18bdba9f43cdb3301a6b1080c70f
I20260809 22:46:42.729759 6061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:42.764338 6061 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:42.765784 6061 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:42.779829 6061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58: Generated new TSK 0
I20260809 22:46:42.780779 6061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab.
Entry for principal oryx/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab.
I20260809 22:46:42.855289 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38065
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab
--principal=oryx/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:43.232288 6079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:43.232581 6079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:43.232723 6079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:43.240087 6079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:43.240208 6079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:43.240303 6079 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:46:43.240386 6079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:43.250547 6079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab
--principal=oryx/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.136.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:43.251931 6079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:43.253269 6079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:43.266933 6090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:43.266896 6094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:43.266953 6091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:43.267705 6079 server_base.cc:1034] running on GCE node
I20260809 22:46:43.268376 6079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:43.269348 6079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:43.270586 6079 hybrid_clock.cc:648] HybridClock initialized: now 1786315603270512 us; error 49 us; skew 500 ppm
I20260809 22:46:43.270915 6079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:43 dist-test-slave-pww0 krb5kdc[5894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315603, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:43.274271 6079 init.cc:377] Logged in from keytab as oryx/127.24.136.1@KRBTEST.COM (short username oryx)
I20260809 22:46:43.275825 6079 webserver.cc:460] Webserver started at http://127.24.136.1:40431/ using document root <none> and password file <none>
I20260809 22:46:43.276712 6079 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:43.276851 6079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:43.277202 6079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:43.281320 6079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "39e1a9218e8a4654a0e10e4e324faa2d"
format_stamp: "Formatted at 2026-08-09 22:46:43 on dist-test-slave-pww0"
server_key: "6eec09e37ba788074dc887ed82831456"
server_key_iv: "cfca64ae2c97eaaabf5fe8e5744c395a"
server_key_version: "encryptionkey@0"
I20260809 22:46:43.282265 6079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "39e1a9218e8a4654a0e10e4e324faa2d"
format_stamp: "Formatted at 2026-08-09 22:46:43 on dist-test-slave-pww0"
server_key: "6eec09e37ba788074dc887ed82831456"
server_key_iv: "cfca64ae2c97eaaabf5fe8e5744c395a"
server_key_version: "encryptionkey@0"
I20260809 22:46:43.288100 6079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:46:43.292075 6106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:43.293612 6079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:46:43.293809 6079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "39e1a9218e8a4654a0e10e4e324faa2d"
format_stamp: "Formatted at 2026-08-09 22:46:43 on dist-test-slave-pww0"
server_key: "6eec09e37ba788074dc887ed82831456"
server_key_iv: "cfca64ae2c97eaaabf5fe8e5744c395a"
server_key_version: "encryptionkey@0"
I20260809 22:46:43.294047 6079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:43.308156 6079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:43.312953 6079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:43.313289 6079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:43.316428 6079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:43.316524 6079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:43.316685 6079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:43.316751 6079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:46:43 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315603, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:43.345166 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:43.317038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:43759 (local address 127.24.136.62:38065)
0809 22:46:43.323862 (+ 6824us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:43.323866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:43.323889 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:46:43.323951 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:43.323955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:43.324036 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:43.324200 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:43.327079 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.328121 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.330962 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.331408 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.331581 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:43.337061 (+ 5480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:43.337069 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:43.337075 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:43.337134 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:43.340669 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:43.341929 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:43.341936 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:43.341940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:43.342092 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:43.342756 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:43.342762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:43.342764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:43.343065 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:43.343222 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:43.344390 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0809 22:46:43.344712 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6698,"thread_start_us":6574,"threads_started":1}
I20260809 22:46:43.346017 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:43.317642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38065 (local address 127.24.136.1:43759)
0809 22:46:43.318556 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:43.318603 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:43.320572 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:43.324609 (+ 4037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:43.324624 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:43.325731 (+ 1107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:43.326814 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:43.326846 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.328339 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.328347 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:43.330724 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:43.330747 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.331744 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.331753 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:43.332143 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:43.333180 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:43.333221 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:43.336688 (+ 3467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:43.340870 (+ 4182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:43.340881 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:43.340898 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:43.341717 (+ 819us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:43.342246 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:43.342251 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:43.342259 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:43.342608 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:43.343345 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:43.343357 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:43.343881 (+ 524us) client_negotiation.cc:769] Sending connection context
0809 22:46:43.344203 (+ 322us) client_negotiation.cc:240] Negotiation successful
0809 22:46:43.344757 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":351,"threads_started":1}
I20260809 22:46:43.368675 6079 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:36889
I20260809 22:46:43.368712 6309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:36889 every 8 connection(s)
I20260809 22:46:43.370070 6079 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:43.371013 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:43.352358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:51905 (local address 127.24.136.62:38065)
0809 22:46:43.352592 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:43.352599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:43.356149 (+ 3550us) server_negotiation.cc:407] Connection header received
0809 22:46:43.356451 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:43.356456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:43.356529 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:43.356632 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:43.358056 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.358956 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.361209 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.361762 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.361908 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:43.364905 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:43.364913 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:43.364919 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:43.364966 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:43.368105 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:43.368948 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:43.368955 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:43.368958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:43.369100 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:43.369484 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:43.369488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:43.369490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:43.369786 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:43.369925 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:43.370399 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 22:46:43.370648 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:46:43.371117 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:43.352270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38065 (local address 127.24.136.1:51905)
0809 22:46:43.356000 (+ 3730us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:43.356022 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:43.356297 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:43.356718 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:43.356723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:43.357428 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:43.357857 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:43.357873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.359166 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.359187 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:43.361006 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:43.361025 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.361997 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.362006 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:43.362244 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:43.363104 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:43.363141 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:43.364592 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:43.368267 (+ 3675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:43.368273 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:43.368277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:43.368747 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:43.369220 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:43.369226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:43.369230 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:43.369365 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:43.369905 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:43.369910 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:43.370098 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 22:46:43.370243 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 22:46:43.370649 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3534,"thread_start_us":3693,"threads_started":1}
I20260809 22:46:43.372926 6310 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38065
I20260809 22:46:43.373212 6310 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:43.373930 6310 heartbeater.cc:511] Master 127.24.136.62:38065 requested a full tablet report, sending...
I20260809 22:46:43.376124 5973 ts_manager.cc:194] Registered new tserver with Master: 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889)
I20260809 22:46:43.377580 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 6079
I20260809 22:46:43.377646 5973 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:43759
I20260809 22:46:43.377763 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:43.378386 25120 external_mini_cluster.cc:1428] Setting key 44c623c9518da22d67e2adc7a8a93e7c
WARNING: no policy specified for oryx/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab.
Entry for principal oryx/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab.
I20260809 22:46:43.435184 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38065
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab
--principal=oryx/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:43.780470 6316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:43.780781 6316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:43.780906 6316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:43.788331 6316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:43.788470 6316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:43.788583 6316 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:46:43.788697 6316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:43.799178 6316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab
--principal=oryx/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.136.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:43.800577 6316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:43.801988 6316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:43.815834 6329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:43.815845 6331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:43.815841 6326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:43.816699 6316 server_base.cc:1034] running on GCE node
I20260809 22:46:43.817395 6316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:43.818414 6316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:43.819643 6316 hybrid_clock.cc:648] HybridClock initialized: now 1786315603819565 us; error 46 us; skew 500 ppm
I20260809 22:46:43.819973 6316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:43 dist-test-slave-pww0 krb5kdc[5894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315603, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:43.823251 6316 init.cc:377] Logged in from keytab as oryx/127.24.136.2@KRBTEST.COM (short username oryx)
I20260809 22:46:43.824810 6316 webserver.cc:460] Webserver started at http://127.24.136.2:44633/ using document root <none> and password file <none>
I20260809 22:46:43.825522 6316 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:43.825614 6316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:43.825902 6316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:43.829094 6316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "3e92eefb1e334928b12d69617e9fc79e"
format_stamp: "Formatted at 2026-08-09 22:46:43 on dist-test-slave-pww0"
server_key: "e32002cc0d9e5c20a47d02231003aba4"
server_key_iv: "dcf278b6c7ee426d73c8df5813a42a54"
server_key_version: "encryptionkey@0"
I20260809 22:46:43.829921 6316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "3e92eefb1e334928b12d69617e9fc79e"
format_stamp: "Formatted at 2026-08-09 22:46:43 on dist-test-slave-pww0"
server_key: "e32002cc0d9e5c20a47d02231003aba4"
server_key_iv: "dcf278b6c7ee426d73c8df5813a42a54"
server_key_version: "encryptionkey@0"
I20260809 22:46:43.835573 6316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 22:46:43.839406 6341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:43.840965 6316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:46:43.841145 6316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "3e92eefb1e334928b12d69617e9fc79e"
format_stamp: "Formatted at 2026-08-09 22:46:43 on dist-test-slave-pww0"
server_key: "e32002cc0d9e5c20a47d02231003aba4"
server_key_iv: "dcf278b6c7ee426d73c8df5813a42a54"
server_key_version: "encryptionkey@0"
I20260809 22:46:43.841358 6316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:43.857465 6316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:43.862353 6316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:43.862609 6316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:43.865538 6316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:43.865645 6316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:43.865811 6316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:43.865875 6316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:46:43 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315603, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:43.892506 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:43.866326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:56759 (local address 127.24.136.62:38065)
0809 22:46:43.866513 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:43.866519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:43.868142 (+ 1623us) server_negotiation.cc:407] Connection header received
0809 22:46:43.870781 (+ 2639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:43.870790 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:43.870921 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:43.871053 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:43.873887 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.875092 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.877443 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.878044 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.878201 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:43.883707 (+ 5506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:43.883716 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:43.883722 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:43.883777 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:43.887716 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:43.889177 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:43.889183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:43.889187 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:43.889372 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:43.890076 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:43.890081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:43.890085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:43.890525 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:43.890700 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:43.891811 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0809 22:46:43.892104 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:46:43.893751 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:43.866944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38065 (local address 127.24.136.2:56759)
0809 22:46:43.867920 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:43.867975 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:43.870357 (+ 2382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:43.871397 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:43.871414 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:43.872482 (+ 1068us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:43.873607 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:43.873636 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.875334 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.875340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:43.877271 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:43.877286 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.878351 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.878363 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:43.878729 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:43.879736 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:43.879771 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:43.883279 (+ 3508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:43.887981 (+ 4702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:43.887997 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:43.888022 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:43.888956 (+ 934us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:43.889541 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:43.889546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:43.889559 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:43.889909 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:43.890711 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:43.890725 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:43.891224 (+ 499us) client_negotiation.cc:769] Sending connection context
0809 22:46:43.891650 (+ 426us) client_negotiation.cc:240] Negotiation successful
0809 22:46:43.892191 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":418,"threads_started":1}
I20260809 22:46:43.909111 6316 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:44329
I20260809 22:46:43.909129 6511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:44329 every 8 connection(s)
I20260809 22:46:43.910545 6316 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:43.912637 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 6316
I20260809 22:46:43.912811 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:43.912899 6513 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38065
I20260809 22:46:43.913157 6513 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:43.913349 25120 external_mini_cluster.cc:1428] Setting key c90a28e627b4760a8e5728093a29818e
I20260809 22:46:43.914057 6513 heartbeater.cc:511] Master 127.24.136.62:38065 requested a full tablet report, sending...
I20260809 22:46:43.915652 5973 ts_manager.cc:194] Registered new tserver with Master: 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329)
I20260809 22:46:43.916507 5973 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:56759
I20260809 22:46:43.917266 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:43.901141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38065 (local address 127.24.136.2:34153)
0809 22:46:43.901983 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:43.902005 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:43.902271 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:43.902878 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:43.902883 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:43.903710 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:43.904266 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:43.904284 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.905912 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.905919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:43.907657 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:43.907675 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.908644 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.908651 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:43.908811 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:43.909445 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:43.909463 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:43.910261 (+ 798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:43.914074 (+ 3813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:43.914080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:43.914084 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:43.914635 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:43.915161 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:43.915167 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:43.915171 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:43.915322 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:43.916141 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:43.916146 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:43.916365 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 22:46:43.916481 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 22:46:43.916802 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":447,"threads_started":1}
I20260809 22:46:43.918215 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:43.901071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:34153 (local address 127.24.136.62:38065)
0809 22:46:43.901252 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:43.901256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:43.902114 (+ 858us) server_negotiation.cc:407] Connection header received
0809 22:46:43.902447 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:43.902454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:43.902547 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:43.902670 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:43.904503 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.905678 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.907830 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:43.908423 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:43.908548 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:43.910487 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:43.910493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:43.910496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:43.910531 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:43.913895 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:43.914816 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:43.914821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:43.914824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:43.915012 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:43.915500 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:43.915504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:43.915507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:43.915974 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:43.917538 (+ 1564us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:43.917666 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 22:46:43.917893 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for oryx/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab.
Entry for principal oryx/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab.
I20260809 22:46:43.972651 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38065
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab
--principal=oryx/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:44.326972 6523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:44.327306 6523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:44.327435 6523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:44.335067 6523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:44.335214 6523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:44.335325 6523 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:46:44.335439 6523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:44.346230 6523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab
--principal=oryx/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.136.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:44.347728 6523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:44.349277 6523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:44.363232 6534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:44.363144 6531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:44.363179 6529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:44.364194 6523 server_base.cc:1034] running on GCE node
I20260809 22:46:44.364840 6523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:44.365866 6523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:44.367098 6523 hybrid_clock.cc:648] HybridClock initialized: now 1786315604367009 us; error 62 us; skew 500 ppm
I20260809 22:46:44.367431 6523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:44 dist-test-slave-pww0 krb5kdc[5894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315604, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:44.371034 6523 init.cc:377] Logged in from keytab as oryx/127.24.136.3@KRBTEST.COM (short username oryx)
I20260809 22:46:44.372897 6523 webserver.cc:460] Webserver started at http://127.24.136.3:46673/ using document root <none> and password file <none>
I20260809 22:46:44.373878 6523 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:44.374022 6523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:44.374387 6523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:44.377496 6523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "83200230b1d247b78488a05038741441"
format_stamp: "Formatted at 2026-08-09 22:46:44 on dist-test-slave-pww0"
server_key: "bb4fd43fd329b0b0db8bdcbf15c9f779"
server_key_iv: "c4fb020fc314acd82f49551ff43ea477"
server_key_version: "encryptionkey@0"
I20260809 22:46:44.378296 6523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "83200230b1d247b78488a05038741441"
format_stamp: "Formatted at 2026-08-09 22:46:44 on dist-test-slave-pww0"
server_key: "bb4fd43fd329b0b0db8bdcbf15c9f779"
server_key_iv: "c4fb020fc314acd82f49551ff43ea477"
server_key_version: "encryptionkey@0"
I20260809 22:46:44.381206 6310 heartbeater.cc:503] Master 127.24.136.62:38065 was elected leader, sending a full tablet report...
I20260809 22:46:44.384450 6523 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260809 22:46:44.388447 6547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:44.390048 6523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:44.390230 6523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "83200230b1d247b78488a05038741441"
format_stamp: "Formatted at 2026-08-09 22:46:44 on dist-test-slave-pww0"
server_key: "bb4fd43fd329b0b0db8bdcbf15c9f779"
server_key_iv: "c4fb020fc314acd82f49551ff43ea477"
server_key_version: "encryptionkey@0"
I20260809 22:46:44.390496 6523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:44.409035 6523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:44.414124 6523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:44.414412 6523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:44.417343 6523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:44.417447 6523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:44.417603 6523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:44.417663 6523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:46:44 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315604, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:44.443118 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:44.418063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:33027 (local address 127.24.136.62:38065)
0809 22:46:44.418270 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:44.418276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:44.419846 (+ 1570us) server_negotiation.cc:407] Connection header received
0809 22:46:44.422813 (+ 2967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:44.422821 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:44.422926 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:44.423064 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:44.425733 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.426924 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.429469 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.429970 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.430104 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:44.435179 (+ 5075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:44.435188 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:44.435193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:44.435267 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:44.438592 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:44.439855 (+ 1263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:44.439862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:44.439866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:44.440171 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:44.440706 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:44.440712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:44.440715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:44.441091 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:44.441218 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:44.442454 (+ 1236us) server_negotiation.cc:299] Negotiation successful
0809 22:46:44.442734 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 22:46:44.444756 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:44.418655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38065 (local address 127.24.136.3:33027)
0809 22:46:44.419625 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:44.419713 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:44.422416 (+ 2703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:44.423603 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:44.423619 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:44.424467 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:44.425475 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:44.425507 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.427129 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.427138 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:44.429287 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:44.429305 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.430255 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.430264 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:44.430629 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:44.431571 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:44.431601 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:44.434812 (+ 3211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:44.438832 (+ 4020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:44.438843 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:44.438868 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:44.439656 (+ 788us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:44.440313 (+ 657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:44.440316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:44.440319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:44.440551 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:44.441336 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:44.441345 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:44.441911 (+ 566us) client_negotiation.cc:769] Sending connection context
0809 22:46:44.442273 (+ 362us) client_negotiation.cc:240] Negotiation successful
0809 22:46:44.442961 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":366,"threads_started":1}
I20260809 22:46:44.461603 6523 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:39061
I20260809 22:46:44.461621 6747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:39061 every 8 connection(s)
I20260809 22:46:44.463073 6523 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:44.465417 6749 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38065
I20260809 22:46:44.465734 6749 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:44.466087 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:44.451506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38065 (local address 127.24.136.3:58699)
0809 22:46:44.452250 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:44.452272 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:44.452451 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:44.452851 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:44.452857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:44.453486 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:44.453898 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:44.453913 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.455023 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.455029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:44.456772 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:44.456794 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.457908 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.457916 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:44.458120 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:44.458851 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:44.458871 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:44.459949 (+ 1078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:44.463119 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:44.463125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:44.463129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:44.463612 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:44.464076 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:44.464082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:44.464085 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:44.464262 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:44.464920 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:44.464925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:44.465139 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 22:46:44.465294 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 22:46:44.465661 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":393,"threads_started":1}
I20260809 22:46:44.466089 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:44.451526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:58699 (local address 127.24.136.62:38065)
0809 22:46:44.451727 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:44.451734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:44.452392 (+ 658us) server_negotiation.cc:407] Connection header received
0809 22:46:44.452569 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:44.452575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:44.452700 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:44.452812 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:44.454081 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.454877 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.456953 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.457490 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.457632 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:44.460160 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:44.460165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:44.460169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:44.460206 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:44.462977 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:44.463747 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:44.463754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:44.463758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:44.463915 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:44.464426 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:44.464431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:44.464434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:44.464797 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:44.464951 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:44.465462 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 22:46:44.465756 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:46:44.466804 6749 heartbeater.cc:511] Master 127.24.136.62:38065 requested a full tablet report, sending...
I20260809 22:46:44.468353 5973 ts_manager.cc:194] Registered new tserver with Master: 83200230b1d247b78488a05038741441 (127.24.136.3:39061)
I20260809 22:46:44.469310 5973 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:33027
I20260809 22:46:44.470912 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 6523
I20260809 22:46:44.471063 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:44.471529 25120 external_mini_cluster.cc:1428] Setting key 9165fe15f9039a9af1a1f6953fe3dd53
I20260809 22:46:44.474977 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:46:44 dist-test-slave-pww0 krb5kdc[5894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315604, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:46:44 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:44 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:44.919776 6513 heartbeater.cc:503] Master 127.24.136.62:38065 was elected leader, sending a full tablet report...
I20260809 22:46:45.472054 6749 heartbeater.cc:503] Master 127.24.136.62:38065 was elected leader, sending a full tablet report...
W20260809 22:46:45.572357 6766 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.042s user 0.002s sys 0.016s
W20260809 22:46:45.573612 6124 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:44.524954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47320 (local address 127.24.136.62:38065)
0809 22:46:44.525207 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:44.525213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:44.525474 (+ 261us) server_negotiation.cc:407] Connection header received
0809 22:46:44.525625 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:44.525629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:44.525707 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:44.525798 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:44.527218 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.528102 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.529402 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:44.529849 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:44.529959 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.573271 (+1043312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47320: BlockingRecv error: recv got EOF from 127.0.0.1:47320 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 22:46:45 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315604, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:45.592980 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.578228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.24.136.62:38065)
0809 22:46:45.578408 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.578413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.578712 (+ 299us) server_negotiation.cc:407] Connection header received
0809 22:46:45.578879 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.578883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.578983 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.579078 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:45.580501 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.581270 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.582694 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.583147 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.583265 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.587324 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:45.587331 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:45.587335 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:45.587379 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:45.590257 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.591066 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.591071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.591073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.591217 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.591695 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.591698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.591701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.592001 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:45.592242 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.592492 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.592689 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:46:45.597611 5973 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:46:45.604962 5973 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:45.629968 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.618279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:44329 (local address 127.0.0.1:56644)
0809 22:46:45.619721 (+ 1442us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.619747 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:45.619870 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:45.621308 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:45.621314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:45.621342 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:45.621785 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.621798 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.623815 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.623820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.626464 (+ 2644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.626483 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.628895 (+ 2412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.628902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.629124 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.629181 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 22:46:45.629332 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 22:46:45.629554 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1293,"thread_start_us":1099,"threads_started":1}
I20260809 22:46:45.631003 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.618943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56644 (local address 127.24.136.2:44329)
0809 22:46:45.620154 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.620161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.620187 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:46:45.620299 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.620306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.620514 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.620700 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:45.621986 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.623613 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.626683 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.628366 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.628507 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.628742 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.630480 (+ 1738us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.630688 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1081,"thread_start_us":1011,"threads_started":1}
I20260809 22:46:45.632738 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.618173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:39061 (local address 127.0.0.1:52564)
0809 22:46:45.618975 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.619025 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:45.619225 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:45.620557 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:45.620564 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:45.620594 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:45.621101 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.621117 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.624551 (+ 3434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.624557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.628273 (+ 3716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.628291 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.631584 (+ 3293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.631591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.632045 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.632085 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 22:46:45.632181 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:46:45.632342 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":355,"threads_started":1}
I20260809 22:46:45.633138 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.618400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.24.136.3:39061)
0809 22:46:45.619892 (+ 1492us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.619901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.619929 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:46:45.620031 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.620039 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.620306 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.620511 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:45.621361 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.624385 (+ 3024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.628550 (+ 4165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.630145 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.630305 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.630521 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.632679 (+ 2158us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.632851 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1340,"thread_start_us":1208,"threads_started":1}
I20260809 22:46:45.636059 6415 tablet_service.cc:1467] Processing CreateTablet for tablet 414b306dcfaa495c92d518c06ef28610 (DEFAULT_TABLE table=test-table [id=1b5d28d712fc4d288f6b310f253ead1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:45.636757 6622 tablet_service.cc:1467] Processing CreateTablet for tablet 414b306dcfaa495c92d518c06ef28610 (DEFAULT_TABLE table=test-table [id=1b5d28d712fc4d288f6b310f253ead1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:45.637683 6415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 414b306dcfaa495c92d518c06ef28610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:45.638051 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 414b306dcfaa495c92d518c06ef28610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:45.641610 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.621080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36889 (local address 127.0.0.1:49756)
0809 22:46:45.631869 (+ 10789us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.631890 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:45.631996 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:45.634479 (+ 2483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:45.634483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:45.634511 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:45.634663 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.634673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.637111 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.637117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.638857 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.638876 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.640792 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.640799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.641001 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.641041 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 22:46:45.641141 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 22:46:45.641310 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10649,"thread_start_us":13129,"threads_started":1}
I20260809 22:46:45.641840 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.621184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.24.136.1:36889)
0809 22:46:45.633852 (+ 12668us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.633859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.633883 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:46:45.633959 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.633965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.634191 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.634402 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:45.634817 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.636950 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.639061 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.640568 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.640746 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.640959 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.641225 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.641410 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12512,"thread_start_us":12353,"threads_started":1}
I20260809 22:46:45.646816 6192 tablet_service.cc:1467] Processing CreateTablet for tablet 414b306dcfaa495c92d518c06ef28610 (DEFAULT_TABLE table=test-table [id=1b5d28d712fc4d288f6b310f253ead1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:45.648423 6192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 414b306dcfaa495c92d518c06ef28610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:45.651887 6792 tablet_bootstrap.cc:492] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: Bootstrap starting.
I20260809 22:46:45.655946 6792 tablet_bootstrap.cc:654] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:45.656225 6791 tablet_bootstrap.cc:492] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: Bootstrap starting.
I20260809 22:46:45.657394 6792 log.cc:824] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:45.660120 6791 tablet_bootstrap.cc:654] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:45.661517 6791 log.cc:824] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:45.664567 6804 tablet_bootstrap.cc:492] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: Bootstrap starting.
I20260809 22:46:45.668529 6804 tablet_bootstrap.cc:654] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:45.669565 6792 tablet_bootstrap.cc:492] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: No bootstrap required, opened a new log
I20260809 22:46:45.669737 6804 log.cc:824] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:45.669839 6792 ts_tablet_manager.cc:1397] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20260809 22:46:45.673516 6804 tablet_bootstrap.cc:492] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: No bootstrap required, opened a new log
I20260809 22:46:45.673792 6804 ts_tablet_manager.cc:1397] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 22:46:45.677724 6791 tablet_bootstrap.cc:492] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: No bootstrap required, opened a new log
I20260809 22:46:45.678082 6791 ts_tablet_manager.cc:1397] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.005s
I20260809 22:46:45.678515 6792 raft_consensus.cc:348] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } }
I20260809 22:46:45.679122 6792 raft_consensus.cc:374] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:45.679284 6792 raft_consensus.cc:729] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83200230b1d247b78488a05038741441, State: Initialized, Role: FOLLOWER
I20260809 22:46:45.680146 6792 consensus_queue.cc:260] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } }
I20260809 22:46:45.684263 6804 raft_consensus.cc:348] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } }
I20260809 22:46:45.684757 6804 raft_consensus.cc:374] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:45.684881 6804 raft_consensus.cc:729] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e1a9218e8a4654a0e10e4e324faa2d, State: Initialized, Role: FOLLOWER
I20260809 22:46:45.686472 6804 consensus_queue.cc:260] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } }
I20260809 22:46:45.686764 6792 ts_tablet_manager.cc:1428] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260809 22:46:45.688493 6804 ts_tablet_manager.cc:1428] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 22:46:45.689591 6791 raft_consensus.cc:348] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } }
I20260809 22:46:45.690145 6791 raft_consensus.cc:374] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:45.690323 6791 raft_consensus.cc:729] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e92eefb1e334928b12d69617e9fc79e, State: Initialized, Role: FOLLOWER
I20260809 22:46:45.690943 6791 consensus_queue.cc:260] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } }
I20260809 22:46:45.692705 6791 ts_tablet_manager.cc:1428] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260809 22:46:45.715771 6751 tablet.cc:2369] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:45.722620 6806 raft_consensus.cc:482] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:45.722927 6806 raft_consensus.cc:504] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } }
I20260809 22:46:45.724740 6806 leader_election.cc:288] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329), 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889)
I20260809 22:46:45.731642 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.725288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:52645 (local address 127.24.136.1:36889)
0809 22:46:45.725498 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.725503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.726236 (+ 733us) server_negotiation.cc:407] Connection header received
0809 22:46:45.726339 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.726345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.726423 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.726491 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:45.727185 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.727981 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.729897 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.730701 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.730854 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.730934 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.731238 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.731373 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:46:45.731647 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.725250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:56025 (local address 127.24.136.2:44329)
0809 22:46:45.725477 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.725483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.726171 (+ 688us) server_negotiation.cc:407] Connection header received
0809 22:46:45.726427 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.726431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.726500 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.726592 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:45.727265 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.728019 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.729873 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.730714 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.730872 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.730947 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.731273 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.731399 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:46:45.731827 6812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.725225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36889 (local address 127.24.136.3:52645)
0809 22:46:45.726057 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.726078 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:45.726207 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:45.726544 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:45.726548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:45.726603 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:45.727032 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.727042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.728112 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.728117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.729730 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.729744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.730927 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.730933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.731083 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.731109 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:46:45.731187 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 22:46:45.731324 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":97,"thread_start_us":449,"threads_started":1}
I20260809 22:46:45.732108 6817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.725216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:44329 (local address 127.24.136.3:56025)
0809 22:46:45.726057 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.726073 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:45.726276 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:45.726622 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:45.726627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:45.726646 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:45.727092 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.727104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.728168 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.728175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.729702 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.729720 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.730981 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.730987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.731107 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.731137 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:46:45.731225 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:46:45.731357 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":39,"spinlock_wait_cycles":27904,"thread_start_us":499,"threads_started":1}
I20260809 22:46:45.732873 6229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "414b306dcfaa495c92d518c06ef28610" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" is_pre_election: true
I20260809 22:46:45.733014 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "414b306dcfaa495c92d518c06ef28610" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e" is_pre_election: true
I20260809 22:46:45.733407 6229 raft_consensus.cc:2454] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 0.
I20260809 22:46:45.733477 6440 raft_consensus.cc:2454] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 0.
I20260809 22:46:45.734076 6553 leader_election.cc:302] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e92eefb1e334928b12d69617e9fc79e, 83200230b1d247b78488a05038741441; no voters:
I20260809 22:46:45.734480 6806 raft_consensus.cc:2781] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:45.734642 6806 raft_consensus.cc:482] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:45.734756 6806 raft_consensus.cc:3037] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:45.737044 6806 raft_consensus.cc:504] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } }
I20260809 22:46:45.737722 6806 leader_election.cc:288] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 election: Requested vote from peers 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329), 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889)
I20260809 22:46:45.738207 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "414b306dcfaa495c92d518c06ef28610" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e"
I20260809 22:46:45.738324 6229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "414b306dcfaa495c92d518c06ef28610" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e1a9218e8a4654a0e10e4e324faa2d"
I20260809 22:46:45.738399 6440 raft_consensus.cc:3037] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:45.738539 6229 raft_consensus.cc:3037] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:45.740666 6440 raft_consensus.cc:2454] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 1.
I20260809 22:46:45.740752 6229 raft_consensus.cc:2454] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 1.
I20260809 22:46:45.741112 6553 leader_election.cc:302] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e92eefb1e334928b12d69617e9fc79e, 83200230b1d247b78488a05038741441; no voters:
I20260809 22:46:45.741421 6806 raft_consensus.cc:2781] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:45.741901 6806 raft_consensus.cc:686] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 1 LEADER]: Becoming Leader. State: Replica: 83200230b1d247b78488a05038741441, State: Running, Role: LEADER
I20260809 22:46:45.742395 6806 consensus_queue.cc:237] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } }
I20260809 22:46:45.747493 5973 catalog_manager.cc:5495] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83200230b1d247b78488a05038741441 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } health_report { overall_health: UNKNOWN } } }
I20260809 22:46:45.789408 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.783827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.24.136.3:39061)
0809 22:46:45.784065 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.784072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.784158 (+ 86us) server_negotiation.cc:407] Connection header received
0809 22:46:45.784411 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.784416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.784500 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.784637 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:45.785057 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.785774 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.787134 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.787596 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.787711 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.788198 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:45.788781 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:45.788923 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.789034 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.789137 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:46:45.798166 6229 raft_consensus.cc:1261] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Refusing update from remote peer 83200230b1d247b78488a05038741441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:45.798182 6440 raft_consensus.cc:1261] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Refusing update from remote peer 83200230b1d247b78488a05038741441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:45.799207 6806 consensus_queue.cc:1035] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:45.799620 6818 consensus_queue.cc:1035] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:45.811933 6823 mvcc.cc:205] Tried to move back new op lower bound from 7316748721338847232 to 7316748721124806656. Current Snapshot: MvccSnapshot[applied={T|T < 7316748721338847232}]
I20260809 22:46:45.827534 5972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:45.827853 5972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:45.831894 5972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58: Sending DeleteTablet for 3 replicas of tablet 414b306dcfaa495c92d518c06ef28610
I20260809 22:46:45.833145 6622 tablet_service.cc:1514] Processing DeleteTablet for tablet 414b306dcfaa495c92d518c06ef28610 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:45 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:52564
I20260809 22:46:45.833118 6192 tablet_service.cc:1514] Processing DeleteTablet for tablet 414b306dcfaa495c92d518c06ef28610 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:45 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:49756
I20260809 22:46:45.833153 6415 tablet_service.cc:1514] Processing DeleteTablet for tablet 414b306dcfaa495c92d518c06ef28610 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:45 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56644
I20260809 22:46:45.835857 5973 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:46:45.836619 5973 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:45.845824 6191 tablet_service.cc:1467] Processing CreateTablet for tablet 28fce611e97a421f81ab3fa6ec4971cd (DEFAULT_TABLE table=test-table [id=a75be4099a444a379b43e96f5c31ca0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:45.845839 6621 tablet_service.cc:1467] Processing CreateTablet for tablet 28fce611e97a421f81ab3fa6ec4971cd (DEFAULT_TABLE table=test-table [id=a75be4099a444a379b43e96f5c31ca0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:45.846495 6191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28fce611e97a421f81ab3fa6ec4971cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:45.846746 6621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28fce611e97a421f81ab3fa6ec4971cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:45.852738 6414 tablet_service.cc:1467] Processing CreateTablet for tablet 28fce611e97a421f81ab3fa6ec4971cd (DEFAULT_TABLE table=test-table [id=a75be4099a444a379b43e96f5c31ca0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:45.853402 6414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28fce611e97a421f81ab3fa6ec4971cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:45.856611 6792 tablet_bootstrap.cc:492] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: Bootstrap starting.
I20260809 22:46:45.858464 6792 tablet_bootstrap.cc:654] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:45.868445 6804 tablet_bootstrap.cc:492] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: Bootstrap starting.
I20260809 22:46:45.868733 6791 tablet_bootstrap.cc:492] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: Bootstrap starting.
I20260809 22:46:45.870262 6804 tablet_bootstrap.cc:654] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:45.870570 6791 tablet_bootstrap.cc:654] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:45.888599 6791 tablet_bootstrap.cc:492] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: No bootstrap required, opened a new log
I20260809 22:46:45.888833 6791 ts_tablet_manager.cc:1397] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.004s
I20260809 22:46:45.889448 6804 tablet_bootstrap.cc:492] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: No bootstrap required, opened a new log
I20260809 22:46:45.889714 6804 ts_tablet_manager.cc:1397] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.011s
I20260809 22:46:45.889823 6791 raft_consensus.cc:348] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:45.890102 6791 raft_consensus.cc:374] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:45.890203 6791 raft_consensus.cc:729] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e92eefb1e334928b12d69617e9fc79e, State: Initialized, Role: FOLLOWER
I20260809 22:46:45.890568 6791 consensus_queue.cc:260] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:45.893250 6792 tablet_bootstrap.cc:492] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: No bootstrap required, opened a new log
I20260809 22:46:45.893911 6804 raft_consensus.cc:348] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:45.894410 6792 ts_tablet_manager.cc:1397] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: Time spent bootstrapping tablet: real 0.038s user 0.012s sys 0.001s
I20260809 22:46:45.894623 6804 raft_consensus.cc:374] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:45.894761 6791 ts_tablet_manager.cc:1428] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260809 22:46:45.896719 6840 tablet_replica.cc:330] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: stopping tablet replica
I20260809 22:46:45.897521 6843 tablet_replica.cc:330] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: stopping tablet replica
I20260809 22:46:45.898042 6843 raft_consensus.cc:2229] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:45.898267 6792 raft_consensus.cc:348] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:45.898517 6792 raft_consensus.cc:374] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:45.898627 6792 raft_consensus.cc:729] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83200230b1d247b78488a05038741441, State: Initialized, Role: FOLLOWER
I20260809 22:46:45.898820 6843 raft_consensus.cc:2258] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:45.898947 6792 consensus_queue.cc:260] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:45.899542 6792 ts_tablet_manager.cc:1428] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 22:46:45.899617 6840 raft_consensus.cc:2229] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:45.902886 6843 ts_tablet_manager.cc:1905] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:45.903558 6840 raft_consensus.cc:2258] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:45.903728 6846 tablet_replica.cc:330] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: stopping tablet replica
I20260809 22:46:45.904197 6846 raft_consensus.cc:2229] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:45.904724 6846 raft_consensus.cc:2258] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:45.894788 6804 raft_consensus.cc:729] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e1a9218e8a4654a0e10e4e324faa2d, State: Initialized, Role: FOLLOWER
I20260809 22:46:45.905550 6804 consensus_queue.cc:260] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:45.906148 6840 ts_tablet_manager.cc:1905] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:45.908742 6804 ts_tablet_manager.cc:1428] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent starting tablet: real 0.019s user 0.000s sys 0.002s
I20260809 22:46:45.912302 6843 ts_tablet_manager.cc:1918] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:45.912513 6843 log.cc:1196] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/414b306dcfaa495c92d518c06ef28610
I20260809 22:46:45.913036 6843 ts_tablet_manager.cc:1939] T 414b306dcfaa495c92d518c06ef28610 P 83200230b1d247b78488a05038741441: Deleting consensus metadata
I20260809 22:46:45.913280 6840 ts_tablet_manager.cc:1918] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:45.913455 6840 log.cc:1196] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/414b306dcfaa495c92d518c06ef28610
W20260809 22:46:45.913738 6517 tablet.cc:2369] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:45.913987 6840 ts_tablet_manager.cc:1939] T 414b306dcfaa495c92d518c06ef28610 P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting consensus metadata
I20260809 22:46:45.915182 5947 catalog_manager.cc:4841] TS 83200230b1d247b78488a05038741441 (127.24.136.3:39061): tablet 414b306dcfaa495c92d518c06ef28610 (table test-table [id=1b5d28d712fc4d288f6b310f253ead1f]) successfully deleted
I20260809 22:46:45.916110 5948 catalog_manager.cc:4841] TS 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889): tablet 414b306dcfaa495c92d518c06ef28610 (table test-table [id=1b5d28d712fc4d288f6b310f253ead1f]) successfully deleted
I20260809 22:46:45.917168 6846 ts_tablet_manager.cc:1905] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:45.924998 6846 ts_tablet_manager.cc:1918] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:45.925223 6846 log.cc:1196] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/414b306dcfaa495c92d518c06ef28610
I20260809 22:46:45.925814 6846 ts_tablet_manager.cc:1939] T 414b306dcfaa495c92d518c06ef28610 P 3e92eefb1e334928b12d69617e9fc79e: Deleting consensus metadata
I20260809 22:46:45.927764 5950 catalog_manager.cc:4841] TS 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329): tablet 414b306dcfaa495c92d518c06ef28610 (table test-table [id=1b5d28d712fc4d288f6b310f253ead1f]) successfully deleted
I20260809 22:46:45.977253 6835 raft_consensus.cc:482] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:45.977521 6835 raft_consensus.cc:504] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:45.978356 6835 leader_election.cc:288] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889), 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329)
I20260809 22:46:45.978964 6229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28fce611e97a421f81ab3fa6ec4971cd" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" is_pre_election: true
I20260809 22:46:45.979001 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28fce611e97a421f81ab3fa6ec4971cd" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e" is_pre_election: true
I20260809 22:46:45.979274 6440 raft_consensus.cc:2454] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 0.
I20260809 22:46:45.979274 6229 raft_consensus.cc:2454] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 0.
I20260809 22:46:45.979775 6550 leader_election.cc:302] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e1a9218e8a4654a0e10e4e324faa2d, 83200230b1d247b78488a05038741441; no voters:
I20260809 22:46:45.980114 6835 raft_consensus.cc:2781] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:45.980280 6835 raft_consensus.cc:482] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:45.980383 6835 raft_consensus.cc:3037] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:45.983733 6835 raft_consensus.cc:504] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:45.984525 6835 leader_election.cc:288] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 election: Requested vote from peers 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889), 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329)
I20260809 22:46:45.985190 6229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28fce611e97a421f81ab3fa6ec4971cd" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e1a9218e8a4654a0e10e4e324faa2d"
I20260809 22:46:45.985215 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28fce611e97a421f81ab3fa6ec4971cd" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e"
I20260809 22:46:45.985427 6229 raft_consensus.cc:3037] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:45.985483 6440 raft_consensus.cc:3037] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:45.987711 6229 raft_consensus.cc:2454] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 1.
I20260809 22:46:45.988291 6550 leader_election.cc:302] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e1a9218e8a4654a0e10e4e324faa2d, 83200230b1d247b78488a05038741441; no voters:
I20260809 22:46:45.988636 6835 raft_consensus.cc:2781] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:45.988811 6835 raft_consensus.cc:686] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 1 LEADER]: Becoming Leader. State: Replica: 83200230b1d247b78488a05038741441, State: Running, Role: LEADER
I20260809 22:46:45.988868 6440 raft_consensus.cc:2454] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 1.
I20260809 22:46:45.989157 6835 consensus_queue.cc:237] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:45.992591 5973 catalog_manager.cc:5495] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83200230b1d247b78488a05038741441 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } health_report { overall_health: HEALTHY } } }
Aug 09 22:46:46 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315602, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.62@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:46.041666 6867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.026119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38065 (local address 127.0.0.1:47352)
0809 22:46:46.026806 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.026826 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.026964 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.027327 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.027330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.028078 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:46.028375 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.028386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.029843 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.029847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.030843 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.030855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.031841 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.031846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.031972 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.032632 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:46.032650 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:46.034896 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:46.038784 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.038800 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.038806 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.039343 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.039845 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.039850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.039852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.040014 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.040734 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:46.040741 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:46.040918 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.041071 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.041323 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":369,"threads_started":1}
I20260809 22:46:46.041967 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.026103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.24.136.62:38065)
0809 22:46:46.026405 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.026412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.026918 (+ 506us) server_negotiation.cc:407] Connection header received
0809 22:46:46.027080 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.027086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.027172 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.027298 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.028541 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.029696 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.030976 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.031661 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.031792 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.035098 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.035105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.035110 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.035148 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.038600 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.039522 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.039529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.039532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.039698 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.040141 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.040146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.040149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.040547 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.040716 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.041235 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.041513 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260809 22:46:46.047811 5973 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:47352:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:46:46.048981 5973 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:46.056628 6192 tablet_service.cc:1467] Processing CreateTablet for tablet 0bef39f98bea4e51b1f35adf8f6e09d9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6eae91ed75544ec98503e6df13e2540d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:46:46.057240 6192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bef39f98bea4e51b1f35adf8f6e09d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.057168 6415 tablet_service.cc:1467] Processing CreateTablet for tablet 0bef39f98bea4e51b1f35adf8f6e09d9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6eae91ed75544ec98503e6df13e2540d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:46:46.057101 6622 tablet_service.cc:1467] Processing CreateTablet for tablet 0bef39f98bea4e51b1f35adf8f6e09d9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6eae91ed75544ec98503e6df13e2540d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:46:46.057760 6415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bef39f98bea4e51b1f35adf8f6e09d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.057765 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bef39f98bea4e51b1f35adf8f6e09d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.064929 6804 tablet_bootstrap.cc:492] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: Bootstrap starting.
I20260809 22:46:46.065393 6792 tablet_bootstrap.cc:492] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: Bootstrap starting.
I20260809 22:46:46.066834 6804 tablet_bootstrap.cc:654] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.067350 6792 tablet_bootstrap.cc:654] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.067760 6791 tablet_bootstrap.cc:492] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: Bootstrap starting.
I20260809 22:46:46.069499 6791 tablet_bootstrap.cc:654] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.069923 6792 tablet_bootstrap.cc:492] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: No bootstrap required, opened a new log
I20260809 22:46:46.070086 6792 ts_tablet_manager.cc:1397] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 22:46:46.070901 6792 raft_consensus.cc:348] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.071218 6792 raft_consensus.cc:374] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.071333 6792 raft_consensus.cc:729] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83200230b1d247b78488a05038741441, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.072171 6792 consensus_queue.cc:260] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.072679 6791 tablet_bootstrap.cc:492] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: No bootstrap required, opened a new log
I20260809 22:46:46.072870 6804 tablet_bootstrap.cc:492] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: No bootstrap required, opened a new log
I20260809 22:46:46.072889 6791 ts_tablet_manager.cc:1397] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260809 22:46:46.072729 6835 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.073097 6835 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:46:46.073268 6804 ts_tablet_manager.cc:1397] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.001s
I20260809 22:46:46.073704 6791 raft_consensus.cc:348] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.073964 6791 raft_consensus.cc:374] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.074051 6791 raft_consensus.cc:729] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e92eefb1e334928b12d69617e9fc79e, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.074626 6791 consensus_queue.cc:260] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.075145 6810 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.075594 6810 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:46:46.076122 6791 ts_tablet_manager.cc:1428] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 22:46:46.077155 6792 ts_tablet_manager.cc:1428] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260809 22:46:46.074393 6804 raft_consensus.cc:348] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.078380 6804 raft_consensus.cc:374] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.078558 6804 raft_consensus.cc:729] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e1a9218e8a4654a0e10e4e324faa2d, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.078922 6804 consensus_queue.cc:260] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.079556 6808 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.079993 6808 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:46:46.080832 6804 ts_tablet_manager.cc:1428] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
W20260809 22:46:46.125453 6311 tablet.cc:2369] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:46.265899 6808 raft_consensus.cc:482] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:46.266194 6808 raft_consensus.cc:504] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.267943 6808 leader_election.cc:288] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83200230b1d247b78488a05038741441 (127.24.136.3:39061), 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329)
I20260809 22:46:46.271163 6835 raft_consensus.cc:482] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:46.271425 6835 raft_consensus.cc:504] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.272315 6835 leader_election.cc:288] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889), 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329)
I20260809 22:46:46.273270 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef39f98bea4e51b1f35adf8f6e09d9" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e" is_pre_election: true
I20260809 22:46:46.273617 6440 raft_consensus.cc:2454] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 0.
I20260809 22:46:46.274190 6553 leader_election.cc:302] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e92eefb1e334928b12d69617e9fc79e, 83200230b1d247b78488a05038741441; no voters:
I20260809 22:46:46.274567 6835 raft_consensus.cc:2781] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:46.274683 6835 raft_consensus.cc:482] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:46.274832 6835 raft_consensus.cc:3037] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:46.278224 6835 raft_consensus.cc:504] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.279817 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef39f98bea4e51b1f35adf8f6e09d9" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e"
I20260809 22:46:46.280078 6440 raft_consensus.cc:3037] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:46.280359 6880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.268350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:44329 (local address 127.24.136.1:53191)
0809 22:46:46.270952 (+ 2602us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.270977 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.271150 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.272322 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.272328 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.272374 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:46.272848 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.272867 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.275113 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.275120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.277612 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.277633 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.279346 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.279354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.279569 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.279610 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.279711 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.279856 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2469,"threads_started":1}
I20260809 22:46:46.282563 6229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef39f98bea4e51b1f35adf8f6e09d9" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" is_pre_election: true
I20260809 22:46:46.282862 6229 raft_consensus.cc:2454] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 0.
I20260809 22:46:46.282804 6228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef39f98bea4e51b1f35adf8f6e09d9" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e1a9218e8a4654a0e10e4e324faa2d"
I20260809 22:46:46.283061 6228 raft_consensus.cc:3037] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:46.283147 6835 leader_election.cc:288] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 election: Requested vote from peers 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889), 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329)
I20260809 22:46:46.284138 6440 raft_consensus.cc:2454] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 1.
I20260809 22:46:46.284667 6553 leader_election.cc:302] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e92eefb1e334928b12d69617e9fc79e, 83200230b1d247b78488a05038741441; no voters:
I20260809 22:46:46.284993 6835 raft_consensus.cc:2781] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:46.285547 6835 raft_consensus.cc:686] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [term 1 LEADER]: Becoming Leader. State: Replica: 83200230b1d247b78488a05038741441, State: Running, Role: LEADER
I20260809 22:46:46.285892 6835 consensus_queue.cc:237] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } }
I20260809 22:46:46.286247 6228 raft_consensus.cc:2454] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 1.
I20260809 22:46:46.288005 6818 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: TxnStatusTablet state changed. Reason: New leader 83200230b1d247b78488a05038741441. Latest consensus state: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.288276 6818 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:46:46.288285 6878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.268439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:53191 (local address 127.24.136.2:44329)
0809 22:46:46.271861 (+ 3422us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.271867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.271891 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:46:46.271974 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.271979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.272102 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.273071 (+ 969us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:46.273271 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.274840 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.277855 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.279132 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.287640 (+ 8508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.287718 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.287777 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.287933 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3185,"thread_start_us":3050,"threads_started":1}
I20260809 22:46:46.289044 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef39f98bea4e51b1f35adf8f6e09d9" candidate_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e" is_pre_election: true
I20260809 22:46:46.289475 6440 raft_consensus.cc:2379] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39e1a9218e8a4654a0e10e4e324faa2d in current term 1: Already voted for candidate 83200230b1d247b78488a05038741441 in this term.
I20260809 22:46:46.289508 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.268523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:57763 (local address 127.24.136.3:39061)
0809 22:46:46.268732 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.268740 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.280861 (+ 12121us) server_negotiation.cc:407] Connection header received
0809 22:46:46.281148 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.281153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.281256 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.281321 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:46.281891 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.282888 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.285657 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.286934 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.287076 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.287165 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.289047 (+ 1882us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.289195 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:46:46.290202 6880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.268590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:39061 (local address 127.24.136.1:57763)
0809 22:46:46.280763 (+ 12173us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.280785 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.280972 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.281518 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.281522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.281551 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:46.281719 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.281730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.283312 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.283318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.285452 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.285468 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.287329 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.287336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.288844 (+ 1508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.288883 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.289733 (+ 850us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.289901 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12047,"thread_start_us":2110,"threads_started":1}
I20260809 22:46:46.290601 5973 catalog_manager.cc:5495] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83200230b1d247b78488a05038741441 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } health_report { overall_health: UNKNOWN } } }
I20260809 22:46:46.291002 6883 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:46:46.291253 6883 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:46:46.291625 6651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bef39f98bea4e51b1f35adf8f6e09d9" candidate_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83200230b1d247b78488a05038741441" is_pre_election: true
I20260809 22:46:46.292780 6107 leader_election.cc:302] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39e1a9218e8a4654a0e10e4e324faa2d; no voters: 3e92eefb1e334928b12d69617e9fc79e, 83200230b1d247b78488a05038741441
I20260809 22:46:46.293314 6808 raft_consensus.cc:2726] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 22:46:46.330650 6867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.325157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:39061 (local address 127.0.0.1:52584)
0809 22:46:46.325399 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.325420 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.325544 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.325955 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.325958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.325968 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:46:46.326129 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.326135 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.327615 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.327619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.328614 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.328627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.329574 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.329578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.329721 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.329806 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:46:46.330210 (+ 404us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:46:46.330221 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.330284 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.330404 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 22:46:46.330744 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.325264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52584 (local address 127.24.136.3:39061)
0809 22:46:46.325492 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.325498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.325523 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:46:46.325727 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.325732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.325836 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.325937 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:46.326274 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.327484 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.328758 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.329428 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.329565 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.329987 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:46.330123 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:46.330219 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.330327 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.330414 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:46:46.334877 6440 raft_consensus.cc:1261] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Refusing update from remote peer 83200230b1d247b78488a05038741441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:46.334941 6228 raft_consensus.cc:1261] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Refusing update from remote peer 83200230b1d247b78488a05038741441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:46.335536 6818 consensus_queue.cc:1035] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:46.336030 6818 consensus_queue.cc:1035] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:46.340385 6808 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: TxnStatusTablet state changed. Reason: New leader 83200230b1d247b78488a05038741441. Latest consensus state: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.340747 6808 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:46:46.344599 6899 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.346429 6899 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:46:46.357249 6808 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.357563 6808 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 39e1a9218e8a4654a0e10e4e324faa2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:46:46.361416 6810 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: TxnStatusTablet state changed. Reason: New leader 83200230b1d247b78488a05038741441. Latest consensus state: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.361756 6810 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:46:46.363327 6899 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.364652 6890 mvcc.cc:205] Tried to move back new op lower bound from 7316748723541245952 to 7316748723350437888. Current Snapshot: MvccSnapshot[applied={T|T < 7316748723541245952}]
I20260809 22:46:46.364806 6899 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 83200230b1d247b78488a05038741441: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:46:46.367095 6810 tablet_replica.cc:439] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } }
I20260809 22:46:46.367357 6810 tablet_replica.cc:442] T 0bef39f98bea4e51b1f35adf8f6e09d9 P 3e92eefb1e334928b12d69617e9fc79e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:46:46.380220 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.372589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53801 (local address 127.24.136.3:39061)
0809 22:46:46.372785 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.372791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.373521 (+ 730us) server_negotiation.cc:407] Connection header received
0809 22:46:46.373716 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.373722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.373804 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.373911 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:46.374435 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.375687 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.378006 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.378700 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.378834 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.379322 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:46.379520 (+ 198us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:46.379605 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.379793 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.379879 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:46:46.380353 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.372683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:39061 (local address 127.24.136.3:53801)
0809 22:46:46.373399 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.373425 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.373581 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.373937 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.373942 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.373954 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:46:46.374280 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.374293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.375851 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.375857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.377843 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.377859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.378885 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.378891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.379064 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.379144 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:46:46.379648 (+ 504us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:46:46.379661 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.379766 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.379913 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":359,"threads_started":1}
I20260809 22:46:46.393455 6228 raft_consensus.cc:1261] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Refusing update from remote peer 83200230b1d247b78488a05038741441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:46.394098 6438 raft_consensus.cc:1261] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Refusing update from remote peer 83200230b1d247b78488a05038741441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:46.394500 6818 consensus_queue.cc:1035] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:46.394820 6835 consensus_queue.cc:1035] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:46.462038 5972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:46.462296 5972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:46.465395 5972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58: Sending DeleteTablet for 3 replicas of tablet 28fce611e97a421f81ab3fa6ec4971cd
I20260809 22:46:46.466321 6192 tablet_service.cc:1514] Processing DeleteTablet for tablet 28fce611e97a421f81ab3fa6ec4971cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:46 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:49756
I20260809 22:46:46.466364 6622 tablet_service.cc:1514] Processing DeleteTablet for tablet 28fce611e97a421f81ab3fa6ec4971cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:46 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:52564
I20260809 22:46:46.466771 6415 tablet_service.cc:1514] Processing DeleteTablet for tablet 28fce611e97a421f81ab3fa6ec4971cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:46 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56644
I20260809 22:46:46.474762 6920 tablet_replica.cc:330] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: stopping tablet replica
I20260809 22:46:46.475636 6920 raft_consensus.cc:2229] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:46.476281 6920 raft_consensus.cc:2258] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:46.477932 6920 ts_tablet_manager.cc:1905] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:46.484366 6918 tablet_replica.cc:330] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: stopping tablet replica
I20260809 22:46:46.484812 6918 raft_consensus.cc:2229] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:46.485424 6918 raft_consensus.cc:2258] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:46.487572 6918 ts_tablet_manager.cc:1905] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:46.487699 6915 tablet_replica.cc:330] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: stopping tablet replica
I20260809 22:46:46.487963 6915 raft_consensus.cc:2229] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:46.488247 6915 raft_consensus.cc:2258] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:46.489651 6915 ts_tablet_manager.cc:1905] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:46.489639 6920 ts_tablet_manager.cc:1918] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:46:46.489784 6920 log.cc:1196] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/28fce611e97a421f81ab3fa6ec4971cd
I20260809 22:46:46.490391 6920 ts_tablet_manager.cc:1939] T 28fce611e97a421f81ab3fa6ec4971cd P 3e92eefb1e334928b12d69617e9fc79e: Deleting consensus metadata
I20260809 22:46:46.491678 5950 catalog_manager.cc:4841] TS 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329): tablet 28fce611e97a421f81ab3fa6ec4971cd (table test-table [id=a75be4099a444a379b43e96f5c31ca0c]) successfully deleted
I20260809 22:46:46.496836 6918 ts_tablet_manager.cc:1918] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:46:46.497119 6918 log.cc:1196] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/28fce611e97a421f81ab3fa6ec4971cd
I20260809 22:46:46.497735 6918 ts_tablet_manager.cc:1939] T 28fce611e97a421f81ab3fa6ec4971cd P 83200230b1d247b78488a05038741441: Deleting consensus metadata
I20260809 22:46:46.499197 5947 catalog_manager.cc:4841] TS 83200230b1d247b78488a05038741441 (127.24.136.3:39061): tablet 28fce611e97a421f81ab3fa6ec4971cd (table test-table [id=a75be4099a444a379b43e96f5c31ca0c]) successfully deleted
I20260809 22:46:46.505436 6915 ts_tablet_manager.cc:1918] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:46:46.505705 6915 log.cc:1196] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/28fce611e97a421f81ab3fa6ec4971cd
I20260809 22:46:46.506389 6915 ts_tablet_manager.cc:1939] T 28fce611e97a421f81ab3fa6ec4971cd P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting consensus metadata
I20260809 22:46:46.507769 5948 catalog_manager.cc:4841] TS 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889): tablet 28fce611e97a421f81ab3fa6ec4971cd (table test-table [id=a75be4099a444a379b43e96f5c31ca0c]) successfully deleted
Aug 09 22:46:46 dist-test-slave-pww0 krb5kdc[5894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:46:46 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:46 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:46:46.549741 6124 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:46.537638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.24.136.62:38065)
0809 22:46:46.537852 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.537859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.538243 (+ 384us) server_negotiation.cc:407] Connection header received
0809 22:46:46.538412 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.538418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.538515 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.538645 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.540122 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.541278 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.542575 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.543295 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.543440 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.549490 (+ 6050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47354: BlockingRecv error: recv got EOF from 127.0.0.1:47354 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 22:46:46 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM
I20260809 22:46:46.568559 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.552580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47370 (local address 127.24.136.62:38065)
0809 22:46:46.552750 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.552754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.553207 (+ 453us) server_negotiation.cc:407] Connection header received
0809 22:46:46.553402 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.553405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.553480 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.553555 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.555210 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.556037 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.557891 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.558315 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.558415 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.563121 (+ 4706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.563126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.563129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.563171 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.565823 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.566550 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.566555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.566557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.566696 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.567107 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.567111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.567113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.567411 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.567696 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.567971 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.568207 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 22:46:46.572482 5972 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:46:46.573094 5972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:46.579881 6192 tablet_service.cc:1467] Processing CreateTablet for tablet 7c94751446ea43d7bb77d0d4589535ba (DEFAULT_TABLE table=test-table [id=4ef5965417c64147a0399cf9242bacd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:46.579957 6415 tablet_service.cc:1467] Processing CreateTablet for tablet 7c94751446ea43d7bb77d0d4589535ba (DEFAULT_TABLE table=test-table [id=4ef5965417c64147a0399cf9242bacd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:46.580077 6622 tablet_service.cc:1467] Processing CreateTablet for tablet 7c94751446ea43d7bb77d0d4589535ba (DEFAULT_TABLE table=test-table [id=4ef5965417c64147a0399cf9242bacd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:46.580394 6192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c94751446ea43d7bb77d0d4589535ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.580399 6415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c94751446ea43d7bb77d0d4589535ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.580476 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c94751446ea43d7bb77d0d4589535ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.588881 6945 tablet_bootstrap.cc:492] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: Bootstrap starting.
I20260809 22:46:46.591289 6945 tablet_bootstrap.cc:654] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.596140 6947 tablet_bootstrap.cc:492] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: Bootstrap starting.
I20260809 22:46:46.598906 6947 tablet_bootstrap.cc:654] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.603325 6945 tablet_bootstrap.cc:492] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: No bootstrap required, opened a new log
I20260809 22:46:46.603497 6945 ts_tablet_manager.cc:1397] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
I20260809 22:46:46.603056 6951 tablet_bootstrap.cc:492] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: Bootstrap starting.
I20260809 22:46:46.604408 6945 raft_consensus.cc:348] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.604585 6947 tablet_bootstrap.cc:492] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: No bootstrap required, opened a new log
I20260809 22:46:46.604672 6945 raft_consensus.cc:374] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.604758 6947 ts_tablet_manager.cc:1397] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260809 22:46:46.604759 6945 raft_consensus.cc:729] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83200230b1d247b78488a05038741441, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.604990 6945 consensus_queue.cc:260] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.605285 6951 tablet_bootstrap.cc:654] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.605646 6945 ts_tablet_manager.cc:1428] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 22:46:46.605755 6947 raft_consensus.cc:348] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.606006 6947 raft_consensus.cc:374] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.606091 6947 raft_consensus.cc:729] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e92eefb1e334928b12d69617e9fc79e, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.606323 6947 consensus_queue.cc:260] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.606892 6947 ts_tablet_manager.cc:1428] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 22:46:46.607954 6951 tablet_bootstrap.cc:492] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: No bootstrap required, opened a new log
I20260809 22:46:46.608122 6951 ts_tablet_manager.cc:1397] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 22:46:46.608915 6951 raft_consensus.cc:348] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.609157 6951 raft_consensus.cc:374] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.609244 6951 raft_consensus.cc:729] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e1a9218e8a4654a0e10e4e324faa2d, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.609462 6951 consensus_queue.cc:260] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.610102 6951 ts_tablet_manager.cc:1428] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:46:46.809486 6808 raft_consensus.cc:482] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:46.809732 6808 raft_consensus.cc:504] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.810545 6808 leader_election.cc:288] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329), 83200230b1d247b78488a05038741441 (127.24.136.3:39061)
I20260809 22:46:46.811127 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c94751446ea43d7bb77d0d4589535ba" candidate_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e" is_pre_election: true
I20260809 22:46:46.811129 6651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c94751446ea43d7bb77d0d4589535ba" candidate_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83200230b1d247b78488a05038741441" is_pre_election: true
I20260809 22:46:46.811400 6440 raft_consensus.cc:2454] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e1a9218e8a4654a0e10e4e324faa2d in term 0.
I20260809 22:46:46.811492 6651 raft_consensus.cc:2454] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e1a9218e8a4654a0e10e4e324faa2d in term 0.
I20260809 22:46:46.812011 6118 leader_election.cc:302] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 39e1a9218e8a4654a0e10e4e324faa2d, 3e92eefb1e334928b12d69617e9fc79e, 83200230b1d247b78488a05038741441; no voters:
I20260809 22:46:46.812359 6808 raft_consensus.cc:2781] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:46.812500 6808 raft_consensus.cc:482] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:46.812592 6808 raft_consensus.cc:3037] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:46.815820 6808 raft_consensus.cc:504] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.816696 6808 leader_election.cc:288] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [CANDIDATE]: Term 1 election: Requested vote from peers 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329), 83200230b1d247b78488a05038741441 (127.24.136.3:39061)
I20260809 22:46:46.817126 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c94751446ea43d7bb77d0d4589535ba" candidate_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e"
I20260809 22:46:46.817386 6440 raft_consensus.cc:3037] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:46.817342 6651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c94751446ea43d7bb77d0d4589535ba" candidate_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83200230b1d247b78488a05038741441"
I20260809 22:46:46.817582 6651 raft_consensus.cc:3037] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:46.820397 6440 raft_consensus.cc:2454] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e1a9218e8a4654a0e10e4e324faa2d in term 1.
I20260809 22:46:46.820730 6651 raft_consensus.cc:2454] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e1a9218e8a4654a0e10e4e324faa2d in term 1.
I20260809 22:46:46.820940 6118 leader_election.cc:302] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e1a9218e8a4654a0e10e4e324faa2d, 3e92eefb1e334928b12d69617e9fc79e; no voters:
I20260809 22:46:46.821348 6808 raft_consensus.cc:2781] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:46.821748 6808 raft_consensus.cc:686] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 LEADER]: Becoming Leader. State: Replica: 39e1a9218e8a4654a0e10e4e324faa2d, State: Running, Role: LEADER
I20260809 22:46:46.822319 6808 consensus_queue.cc:237] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.826419 5973 catalog_manager.cc:5495] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1). New cstate: current_term: 1 leader_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } health_report { overall_health: UNKNOWN } } }
I20260809 22:46:46.865530 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.858409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49764 (local address 127.24.136.1:36889)
0809 22:46:46.859148 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.859155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.859178 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:46:46.859305 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.859310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.859431 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.859658 (+ 227us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:46.860043 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.861206 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.862587 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.863273 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.863397 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.863918 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:46.864754 (+ 836us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:46.864908 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.865033 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.865140 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":421,"threads_started":1}
I20260809 22:46:46.869472 6440 raft_consensus.cc:1261] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Refusing update from remote peer 39e1a9218e8a4654a0e10e4e324faa2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:46.869669 6651 raft_consensus.cc:1261] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Refusing update from remote peer 39e1a9218e8a4654a0e10e4e324faa2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:46.870321 6954 consensus_queue.cc:1035] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:46.870721 6808 consensus_queue.cc:1035] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:46.875880 6893 mvcc.cc:205] Tried to move back new op lower bound from 7316748725729583104 to 7316748725547937792. Current Snapshot: MvccSnapshot[applied={T|T < 7316748725729583104}]
I20260809 22:46:46.893914 5972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:46.894254 5972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:46.898440 5972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58: Sending DeleteTablet for 3 replicas of tablet 7c94751446ea43d7bb77d0d4589535ba
I20260809 22:46:46.899560 6622 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c94751446ea43d7bb77d0d4589535ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:46 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:52564
I20260809 22:46:46.899878 6918 tablet_replica.cc:330] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: stopping tablet replica
I20260809 22:46:46.899941 6192 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c94751446ea43d7bb77d0d4589535ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:46 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:49756
I20260809 22:46:46.900086 6918 raft_consensus.cc:2229] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:46.900226 6915 tablet_replica.cc:330] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: stopping tablet replica
I20260809 22:46:46.900363 6918 raft_consensus.cc:2258] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:46.900431 6915 raft_consensus.cc:2229] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:46.900722 6415 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c94751446ea43d7bb77d0d4589535ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:46 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56644
I20260809 22:46:46.900943 6915 raft_consensus.cc:2258] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:46.901230 6920 tablet_replica.cc:330] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: stopping tablet replica
I20260809 22:46:46.901417 6920 raft_consensus.cc:2229] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:46.901587 6920 raft_consensus.cc:2258] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:46.902391 5972 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 22:46:46.902688 6915 ts_tablet_manager.cc:1905] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:46.902911 6918 ts_tablet_manager.cc:1905] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 22:46:46.903096 5972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:46.909766 6915 ts_tablet_manager.cc:1918] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:46.909998 6915 log.cc:1196] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/7c94751446ea43d7bb77d0d4589535ba
I20260809 22:46:46.910158 6918 ts_tablet_manager.cc:1918] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:46.910378 6918 log.cc:1196] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/7c94751446ea43d7bb77d0d4589535ba
I20260809 22:46:46.910475 6915 ts_tablet_manager.cc:1939] T 7c94751446ea43d7bb77d0d4589535ba P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting consensus metadata
I20260809 22:46:46.910853 6918 ts_tablet_manager.cc:1939] T 7c94751446ea43d7bb77d0d4589535ba P 83200230b1d247b78488a05038741441: Deleting consensus metadata
I20260809 22:46:46.911994 5947 catalog_manager.cc:4841] TS 83200230b1d247b78488a05038741441 (127.24.136.3:39061): tablet 7c94751446ea43d7bb77d0d4589535ba (table test-table [id=4ef5965417c64147a0399cf9242bacd7]) successfully deleted
I20260809 22:46:46.912380 5948 catalog_manager.cc:4841] TS 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889): tablet 7c94751446ea43d7bb77d0d4589535ba (table test-table [id=4ef5965417c64147a0399cf9242bacd7]) successfully deleted
I20260809 22:46:46.912858 6622 tablet_service.cc:1467] Processing CreateTablet for tablet 863c87b99c004bf1946df38ec350c741 (DEFAULT_TABLE table=test-table [id=cc487337239f45ae84071d3657e2cf43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:46.913105 6192 tablet_service.cc:1467] Processing CreateTablet for tablet 863c87b99c004bf1946df38ec350c741 (DEFAULT_TABLE table=test-table [id=cc487337239f45ae84071d3657e2cf43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:46.913264 6622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 863c87b99c004bf1946df38ec350c741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.913518 6192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 863c87b99c004bf1946df38ec350c741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.916487 6415 tablet_service.cc:1467] Processing CreateTablet for tablet 863c87b99c004bf1946df38ec350c741 (DEFAULT_TABLE table=test-table [id=cc487337239f45ae84071d3657e2cf43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:46.917078 6415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 863c87b99c004bf1946df38ec350c741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.919873 6945 tablet_bootstrap.cc:492] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: Bootstrap starting.
I20260809 22:46:46.921044 6920 ts_tablet_manager.cc:1905] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:46.921545 6945 tablet_bootstrap.cc:654] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.922816 6951 tablet_bootstrap.cc:492] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: Bootstrap starting.
I20260809 22:46:46.924597 6951 tablet_bootstrap.cc:654] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.925788 6947 tablet_bootstrap.cc:492] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: Bootstrap starting.
I20260809 22:46:46.927642 6947 tablet_bootstrap.cc:654] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.927678 6920 ts_tablet_manager.cc:1918] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:46.927824 6920 log.cc:1196] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/7c94751446ea43d7bb77d0d4589535ba
I20260809 22:46:46.928263 6920 ts_tablet_manager.cc:1939] T 7c94751446ea43d7bb77d0d4589535ba P 3e92eefb1e334928b12d69617e9fc79e: Deleting consensus metadata
I20260809 22:46:46.929414 5950 catalog_manager.cc:4841] TS 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329): tablet 7c94751446ea43d7bb77d0d4589535ba (table test-table [id=4ef5965417c64147a0399cf9242bacd7]) successfully deleted
I20260809 22:46:46.936295 6951 tablet_bootstrap.cc:492] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: No bootstrap required, opened a new log
I20260809 22:46:46.936492 6951 ts_tablet_manager.cc:1397] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260809 22:46:46.937346 6951 raft_consensus.cc:348] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.937644 6951 raft_consensus.cc:374] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.937723 6951 raft_consensus.cc:729] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e1a9218e8a4654a0e10e4e324faa2d, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.938009 6951 consensus_queue.cc:260] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.938658 6951 ts_tablet_manager.cc:1428] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 22:46:46.944620 6947 tablet_bootstrap.cc:492] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: No bootstrap required, opened a new log
I20260809 22:46:46.944962 6947 ts_tablet_manager.cc:1397] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.003s
I20260809 22:46:46.947374 6947 raft_consensus.cc:348] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.947839 6947 raft_consensus.cc:374] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.947981 6947 raft_consensus.cc:729] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e92eefb1e334928b12d69617e9fc79e, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.948519 6947 consensus_queue.cc:260] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.951640 6947 ts_tablet_manager.cc:1428] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260809 22:46:46.956763 6945 tablet_bootstrap.cc:492] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: No bootstrap required, opened a new log
I20260809 22:46:46.956979 6945 ts_tablet_manager.cc:1397] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: Time spent bootstrapping tablet: real 0.037s user 0.008s sys 0.008s
I20260809 22:46:46.957852 6945 raft_consensus.cc:348] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.958148 6945 raft_consensus.cc:374] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.958240 6945 raft_consensus.cc:729] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83200230b1d247b78488a05038741441, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.958504 6945 consensus_queue.cc:260] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:46.959077 6945 ts_tablet_manager.cc:1428] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:46:47.043016 6913 raft_consensus.cc:482] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:47.043278 6913 raft_consensus.cc:504] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:47.044078 6913 leader_election.cc:288] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889), 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329)
I20260809 22:46:47.044610 6228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "863c87b99c004bf1946df38ec350c741" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" is_pre_election: true
I20260809 22:46:47.044852 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "863c87b99c004bf1946df38ec350c741" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e" is_pre_election: true
I20260809 22:46:47.044973 6228 raft_consensus.cc:2454] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 0.
I20260809 22:46:47.045145 6440 raft_consensus.cc:2454] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 0.
I20260809 22:46:47.045639 6550 leader_election.cc:302] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 39e1a9218e8a4654a0e10e4e324faa2d, 3e92eefb1e334928b12d69617e9fc79e, 83200230b1d247b78488a05038741441; no voters:
I20260809 22:46:47.045966 6913 raft_consensus.cc:2781] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:47.046097 6913 raft_consensus.cc:482] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:47.046185 6913 raft_consensus.cc:3037] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:47.048518 6913 raft_consensus.cc:504] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:47.049160 6913 leader_election.cc:288] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 election: Requested vote from peers 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889), 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329)
I20260809 22:46:47.049757 6228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "863c87b99c004bf1946df38ec350c741" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e1a9218e8a4654a0e10e4e324faa2d"
I20260809 22:46:47.049983 6228 raft_consensus.cc:3037] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:47.052332 6228 raft_consensus.cc:2454] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 1.
I20260809 22:46:47.052780 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "863c87b99c004bf1946df38ec350c741" candidate_uuid: "83200230b1d247b78488a05038741441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e92eefb1e334928b12d69617e9fc79e"
I20260809 22:46:47.052851 6550 leader_election.cc:302] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e1a9218e8a4654a0e10e4e324faa2d, 83200230b1d247b78488a05038741441; no voters:
I20260809 22:46:47.053057 6440 raft_consensus.cc:3037] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:47.053194 6913 raft_consensus.cc:2781] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:47.053654 6913 raft_consensus.cc:686] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 1 LEADER]: Becoming Leader. State: Replica: 83200230b1d247b78488a05038741441, State: Running, Role: LEADER
I20260809 22:46:47.054031 6913 consensus_queue.cc:237] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } }
I20260809 22:46:47.056226 6440 raft_consensus.cc:2454] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83200230b1d247b78488a05038741441 in term 1.
I20260809 22:46:47.058883 5972 catalog_manager.cc:5495] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83200230b1d247b78488a05038741441 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "83200230b1d247b78488a05038741441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83200230b1d247b78488a05038741441" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39061 } health_report { overall_health: HEALTHY } } }
I20260809 22:46:47.090368 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:47.084409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.24.136.3:39061)
0809 22:46:47.085003 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:47.085007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:47.085027 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:47.085084 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:47.085087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:47.085151 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:47.085230 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:47.085673 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:47.086408 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:47.088452 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:47.088900 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:47.089011 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:47.089547 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:47.089698 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:47.089805 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:47.089958 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 22:46:47.090031 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":365,"threads_started":1}
I20260809 22:46:47.101007 6440 raft_consensus.cc:1261] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Refusing update from remote peer 83200230b1d247b78488a05038741441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:47.101142 6228 raft_consensus.cc:1261] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Refusing update from remote peer 83200230b1d247b78488a05038741441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:47.101627 6973 consensus_queue.cc:1035] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e92eefb1e334928b12d69617e9fc79e" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:47.102046 6968 consensus_queue.cc:1035] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39e1a9218e8a4654a0e10e4e324faa2d" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:47.203187 5972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:47.203497 5972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:47.207801 5972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe2005c200ca4d599ce79202e56f3b58: Sending DeleteTablet for 3 replicas of tablet 863c87b99c004bf1946df38ec350c741
I20260809 22:46:47.209010 6192 tablet_service.cc:1514] Processing DeleteTablet for tablet 863c87b99c004bf1946df38ec350c741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:47 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:49756
I20260809 22:46:47.209384 6915 tablet_replica.cc:330] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: stopping tablet replica
I20260809 22:46:47.209596 6915 raft_consensus.cc:2229] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:47.209848 6915 raft_consensus.cc:2258] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:47.210014 6415 tablet_service.cc:1514] Processing DeleteTablet for tablet 863c87b99c004bf1946df38ec350c741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:47 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56644
I20260809 22:46:47.210384 6920 tablet_replica.cc:330] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: stopping tablet replica
I20260809 22:46:47.210523 6920 raft_consensus.cc:2229] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:47.210683 6920 raft_consensus.cc:2258] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:47.211181 6915 ts_tablet_manager.cc:1905] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:47.211753 6920 ts_tablet_manager.cc:1905] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:47.216439 6621 tablet_service.cc:1514] Processing DeleteTablet for tablet 863c87b99c004bf1946df38ec350c741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:47 UTC) from {username='oryx', principal='oryx/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:52564
I20260809 22:46:47.216807 6918 tablet_replica.cc:330] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: stopping tablet replica
I20260809 22:46:47.217027 6918 raft_consensus.cc:2229] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:47.217437 6918 raft_consensus.cc:2258] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:47.218701 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 6079
I20260809 22:46:47.219125 6920 ts_tablet_manager.cc:1918] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:46:47.219292 6920 log.cc:1196] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/863c87b99c004bf1946df38ec350c741
I20260809 22:46:47.219741 6920 ts_tablet_manager.cc:1939] T 863c87b99c004bf1946df38ec350c741 P 3e92eefb1e334928b12d69617e9fc79e: Deleting consensus metadata
I20260809 22:46:47.221273 6918 ts_tablet_manager.cc:1905] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:47.223721 5950 catalog_manager.cc:4841] TS 3e92eefb1e334928b12d69617e9fc79e (127.24.136.2:44329): tablet 863c87b99c004bf1946df38ec350c741 (table test-table [id=cc487337239f45ae84071d3657e2cf43]) successfully deleted
I20260809 22:46:47.230365 6918 ts_tablet_manager.cc:1918] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
Aug 09 22:46:47 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.1@KRBTEST.COM
I20260809 22:46:47.233494 6915 ts_tablet_manager.cc:1918] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:46:47.233843 6918 log.cc:1196] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/863c87b99c004bf1946df38ec350c741
I20260809 22:46:47.234431 6918 ts_tablet_manager.cc:1939] T 863c87b99c004bf1946df38ec350c741 P 83200230b1d247b78488a05038741441: Deleting consensus metadata
I20260809 22:46:47.234659 6915 log.cc:1196] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/863c87b99c004bf1946df38ec350c741
I20260809 22:46:47.236337 5947 catalog_manager.cc:4841] TS 83200230b1d247b78488a05038741441 (127.24.136.3:39061): tablet 863c87b99c004bf1946df38ec350c741 (table test-table [id=cc487337239f45ae84071d3657e2cf43]) successfully deleted
I20260809 22:46:47.236593 6915 ts_tablet_manager.cc:1939] T 863c87b99c004bf1946df38ec350c741 P 39e1a9218e8a4654a0e10e4e324faa2d: Deleting consensus metadata
I20260809 22:46:47.237967 5948 catalog_manager.cc:4841] TS 39e1a9218e8a4654a0e10e4e324faa2d (127.24.136.1:36889): tablet 863c87b99c004bf1946df38ec350c741 (table test-table [id=cc487337239f45ae84071d3657e2cf43]) successfully deleted
I20260809 22:46:47.241952 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:47.220134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49772 (local address 127.24.136.1:36889)
0809 22:46:47.220333 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:47.220340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:47.220445 (+ 105us) server_negotiation.cc:407] Connection header received
0809 22:46:47.220740 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:47.220746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:47.220828 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:47.220910 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:47.222751 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:47.224008 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:47.225956 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:47.226650 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:47.226766 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:47.232796 (+ 6030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:47.232819 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:47.232842 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:47.232903 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:47.237732 (+ 4829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:47.238853 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:47.238864 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:47.238867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:47.239055 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:47.239634 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:47.239639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:47.239643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:47.240196 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:47.240526 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:47.241201 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 22:46:47.241496 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:46:47.243547 6304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:47.375356 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 6079
I20260809 22:46:47.403155 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 6316
Aug 09 22:46:47 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.2@KRBTEST.COM
I20260809 22:46:47.419646 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:47.404010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56654 (local address 127.24.136.2:44329)
0809 22:46:47.404519 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:47.404523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:47.404538 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:46:47.404592 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:47.404595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:47.404654 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:47.404757 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:47.406127 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:47.406908 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:47.408220 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:47.408640 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:47.408764 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:47.413166 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:47.413187 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:47.413205 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:47.413264 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:47.416427 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:47.417277 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:47.417285 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:47.417288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:47.417445 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:47.417853 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:47.417857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:47.417859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:47.418338 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:47.418667 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:47.418990 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 22:46:47.419233 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":244,"threads_started":1}
I20260809 22:46:47.420353 6505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:47.542663 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 6316
I20260809 22:46:47.571668 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 6523
Aug 09 22:46:47 dist-test-slave-pww0 krb5kdc[5894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.136.3@KRBTEST.COM
I20260809 22:46:47.587854 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:47.572465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.24.136.3:39061)
0809 22:46:47.572666 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:47.572671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:47.572691 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:47.572877 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:47.572881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:47.572963 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:47.573052 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:47.574407 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:47.575153 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:47.576563 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:47.576980 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:47.577097 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:47.581633 (+ 4536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:47.581652 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:47.581671 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:47.581714 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:47.584746 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:47.585520 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:47.585530 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:47.585532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:47.585695 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:47.586131 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:47.586136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:47.586138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:47.586547 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:47.586849 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:47.587227 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 22:46:47.587521 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:46:47.588624 6741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:47.749369 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 6523
I20260809 22:46:47.787746 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 5921
I20260809 22:46:47.789012 6031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:47.907684 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 5921
2026-08-09T22:46:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7253 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:48 dist-test-slave-pww0 krb5kdc[7006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:48 dist-test-slave-pww0 krb5kdc[7006](info): set up 2 sockets
Aug 09 22:46:48 dist-test-slave-pww0 krb5kdc[7006](info): commencing operation
krb5kdc: starting...
W20260809 22:46:49.195807 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:46:49 dist-test-slave-pww0 krb5kdc[7006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:46:49.487531 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:42767
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:38723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:42767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:49.849997 7025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:49.850338 7025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:49.850446 7025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:49.858227 7025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:49.858351 7025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:49.858395 7025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:49.858451 7025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:49.858498 7025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:49.868700 7025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:38723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:42767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:42767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:49.870052 7025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:49.871371 7025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:49.883925 7030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:49.883876 7032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:49.884138 7025 server_base.cc:1034] running on GCE node
W20260809 22:46:49.883876 7037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:49.885365 7025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:49.887171 7025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:49.888377 7025 hybrid_clock.cc:648] HybridClock initialized: now 1786315609888304 us; error 60 us; skew 500 ppm
I20260809 22:46:49.888698 7025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T22:46:49Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2045 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:50 dist-test-slave-pww0 krb5kdc[7044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:50 dist-test-slave-pww0 krb5kdc[7044](info): set up 2 sockets
Aug 09 22:46:50 dist-test-slave-pww0 krb5kdc[7044](info): commencing operation
krb5kdc: starting...
W20260809 22:46:51.231665 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:46:51 dist-test-slave-pww0 krb5kdc[7044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:46:51.518839 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38915
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:51.873206 7069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:51.873548 7069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:51.873639 7069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:51.881235 7069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:51.881371 7069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:51.881419 7069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:51.881475 7069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:51.881520 7069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:51.891922 7069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:51.893338 7069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:51.894708 7069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:51.907569 7079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:51.907663 7077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:51.907608 7085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:51.907864 7069 server_base.cc:1034] running on GCE node
I20260809 22:46:51.909235 7069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:51.910876 7069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:51.912148 7069 hybrid_clock.cc:648] HybridClock initialized: now 1786315611912069 us; error 50 us; skew 500 ppm
I20260809 22:46:51.912544 7069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:51 dist-test-slave-pww0 krb5kdc[7044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:51.916211 7069 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:46:51.917935 7069 webserver.cc:460] Webserver started at http://127.24.136.62:40141/ using document root <none> and password file <none>
I20260809 22:46:51.918723 7069 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:51.918859 7069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:51.919198 7069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:51.922498 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "75b3711de8af429c9c8e249518c9af72"
format_stamp: "Formatted at 2026-08-09 22:46:51 on dist-test-slave-pww0"
server_key: "96a224a94ff63d83636913cf9d23f8ef"
server_key_iv: "c7a35bd7e127c4593efa5214a17b58be"
server_key_version: "encryptionkey@0"
I20260809 22:46:51.923377 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "75b3711de8af429c9c8e249518c9af72"
format_stamp: "Formatted at 2026-08-09 22:46:51 on dist-test-slave-pww0"
server_key: "96a224a94ff63d83636913cf9d23f8ef"
server_key_iv: "c7a35bd7e127c4593efa5214a17b58be"
server_key_version: "encryptionkey@0"
I20260809 22:46:51.929509 7069 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 22:46:51.934190 7096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:51.935891 7069 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260809 22:46:51.936115 7069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "75b3711de8af429c9c8e249518c9af72"
format_stamp: "Formatted at 2026-08-09 22:46:51 on dist-test-slave-pww0"
server_key: "96a224a94ff63d83636913cf9d23f8ef"
server_key_iv: "c7a35bd7e127c4593efa5214a17b58be"
server_key_version: "encryptionkey@0"
I20260809 22:46:51.936340 7069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:51.961110 7069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:51.965972 7069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:51.966243 7069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:51.984476 7069 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38915
I20260809 22:46:51.984508 7182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38915 every 8 connection(s)
I20260809 22:46:51.985893 7069 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:51.987483 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 7069
I20260809 22:46:51.987744 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:46:51.988601 25120 external_mini_cluster.cc:1428] Setting key bc880e8365dc17a9494339e5b709d2c5
I20260809 22:46:51.991058 7184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:52.002961 7184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: Bootstrap starting.
Aug 09 22:46:51 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:52.006649 7184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:52.007854 7184 log.cc:824] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:52.011620 7184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: No bootstrap required, opened a new log
I20260809 22:46:52.013113 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:51.990930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60550 (local address 127.24.136.62:38915)
0809 22:46:51.991823 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:51.991836 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:51.991881 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:46:51.993688 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:51.993714 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:51.994141 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:51.994620 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:51.996310 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:51.997684 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:51.998983 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:51.999791 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:51.999977 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:52.004428 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:52.004438 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:52.004455 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:52.004523 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:52.007946 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:52.008900 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:52.008908 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:52.008915 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:52.009124 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:52.009580 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:52.009585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:52.009588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:52.010376 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:52.010686 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:52.011151 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 22:46:52.011664 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":400,"threads_started":1}
I20260809 22:46:52.021133 7184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } }
I20260809 22:46:52.021556 7184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:52.021644 7184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b3711de8af429c9c8e249518c9af72, State: Initialized, Role: FOLLOWER
I20260809 22:46:52.022182 7184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } }
I20260809 22:46:52.022394 7184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:52.022497 7184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:52.022639 7184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:52.025218 7184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } }
I20260809 22:46:52.025770 7184 leader_election.cc:302] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b3711de8af429c9c8e249518c9af72; no voters:
I20260809 22:46:52.026396 7184 leader_election.cc:288] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:52.026557 7192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:52.027220 7192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 1 LEADER]: Becoming Leader. State: Replica: 75b3711de8af429c9c8e249518c9af72, State: Running, Role: LEADER
I20260809 22:46:52.027854 7192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } }
I20260809 22:46:52.028524 7184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:52.030805 7194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b3711de8af429c9c8e249518c9af72. Latest consensus state: current_term: 1 leader_uuid: "75b3711de8af429c9c8e249518c9af72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } } }
I20260809 22:46:52.031100 7193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75b3711de8af429c9c8e249518c9af72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } } }
I20260809 22:46:52.031351 7194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:52.031430 7193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:52.032414 7204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:52.038913 7204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:52.049741 7204 catalog_manager.cc:1347] Generated new cluster ID: 5ef5d603ae034142934c82920864fcd3
I20260809 22:46:52.049886 7204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:52.072152 7204 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:52.073565 7204 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:52.080763 7204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: Generated new TSK 0
I20260809 22:46:52.081626 7204 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:46:52.157253 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38915
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:52.511235 7226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:52.511600 7226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:52.511754 7226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:52.519536 7226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:52.519666 7226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:52.519822 7226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:52.530330 7226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:52.531858 7226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:52.533393 7226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:52.547850 7234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:52.547931 7239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:52.547900 7235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:52.548766 7226 server_base.cc:1034] running on GCE node
I20260809 22:46:52.549431 7226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:52.550490 7226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:52.551752 7226 hybrid_clock.cc:648] HybridClock initialized: now 1786315612551674 us; error 71 us; skew 500 ppm
I20260809 22:46:52.552114 7226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:52 dist-test-slave-pww0 krb5kdc[7044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315612, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:52.555958 7226 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:46:52.557571 7226 webserver.cc:460] Webserver started at http://127.24.136.1:43735/ using document root <none> and password file <none>
I20260809 22:46:52.558315 7226 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:52.558431 7226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:52.558717 7226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:52.561921 7226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "8defd5d77d7943ef8206ae87134fb444"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-pww0"
server_key: "2c08c31e23b39d2c6316190bcee187ea"
server_key_iv: "c414efafa216482455dc497f408b9c2e"
server_key_version: "encryptionkey@0"
I20260809 22:46:52.562938 7226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "8defd5d77d7943ef8206ae87134fb444"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-pww0"
server_key: "2c08c31e23b39d2c6316190bcee187ea"
server_key_iv: "c414efafa216482455dc497f408b9c2e"
server_key_version: "encryptionkey@0"
I20260809 22:46:52.569044 7226 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 22:46:52.573170 7247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:52.574869 7226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:46:52.575093 7226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "8defd5d77d7943ef8206ae87134fb444"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-pww0"
server_key: "2c08c31e23b39d2c6316190bcee187ea"
server_key_iv: "c414efafa216482455dc497f408b9c2e"
server_key_version: "encryptionkey@0"
I20260809 22:46:52.575344 7226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:52.605351 7226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:52.610168 7226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:52.610512 7226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:52.611625 7226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:52.613108 7226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:52.613195 7226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:52.613327 7226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:52.613392 7226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:52.646023 7226 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:45569
I20260809 22:46:52.646211 7468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:45569 every 8 connection(s)
I20260809 22:46:52.647595 7226 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:52.656818 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 7226
I20260809 22:46:52.657006 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:52.657737 25120 external_mini_cluster.cc:1428] Setting key 0622e9340999b706493c3321e4cbadc0
Aug 09 22:46:52 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315612, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:52.671080 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:52.650717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:47577 (local address 127.24.136.62:38915)
0809 22:46:52.651223 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:52.651228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:52.652252 (+ 1024us) server_negotiation.cc:407] Connection header received
0809 22:46:52.654383 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:52.654389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:52.654473 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:52.654577 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:52.656505 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.657369 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.659639 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.660260 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.660385 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:52.664947 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:52.664954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:52.664957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:52.664999 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:52.667706 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:52.668563 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:52.668568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:52.668571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:52.668716 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:52.669181 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:52.669186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:52.669189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:52.669473 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:52.669629 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:52.670451 (+ 822us) server_negotiation.cc:299] Negotiation successful
0809 22:46:52.670723 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":264,"threads_started":1}
I20260809 22:46:52.672179 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:52.651216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:47577)
0809 22:46:52.652047 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:52.652098 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:52.653959 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:52.654883 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:52.654893 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:52.655567 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:52.656245 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:52.656270 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.657599 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.657605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:52.659418 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:52.659436 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.660574 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.660583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:52.660942 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:52.661807 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:52.661841 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:52.664638 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:52.667901 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:52.667910 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:52.667925 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:52.668415 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:52.668831 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:52.668836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:52.668840 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:52.669056 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:52.669608 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:52.669617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:52.670008 (+ 391us) client_negotiation.cc:769] Sending connection context
0809 22:46:52.670302 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 22:46:52.670904 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":350,"threads_started":1}
I20260809 22:46:52.674094 7472 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38915
I20260809 22:46:52.674715 7472 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:52.675783 7472 heartbeater.cc:511] Master 127.24.136.62:38915 requested a full tablet report, sending...
I20260809 22:46:52.678902 7128 ts_manager.cc:194] Registered new tserver with Master: 8defd5d77d7943ef8206ae87134fb444 (127.24.136.1:45569)
I20260809 22:46:52.681072 7128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:47577
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:46:52.723783 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38915
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:53.080112 7479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:53.080416 7479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:53.080511 7479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:53.088111 7479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:53.088240 7479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:53.088392 7479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:53.098590 7479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:53.100081 7479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:53.101459 7479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:53.115020 7494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:53.114995 7490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:53.115216 7489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:53.115893 7479 server_base.cc:1034] running on GCE node
I20260809 22:46:53.116554 7479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:53.117610 7479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:53.118819 7479 hybrid_clock.cc:648] HybridClock initialized: now 1786315613118754 us; error 63 us; skew 500 ppm
I20260809 22:46:53.119158 7479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:53 dist-test-slave-pww0 krb5kdc[7044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315613, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:53.128842 7479 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:46:53.131173 7479 webserver.cc:460] Webserver started at http://127.24.136.2:33545/ using document root <none> and password file <none>
I20260809 22:46:53.132460 7479 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:53.132637 7479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:53.133067 7479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:53.136547 7479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "4395dd876cdc4b21b6f7d8c30c28715e"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-pww0"
server_key: "814bd653f45686659a6480aefa5df6b9"
server_key_iv: "f104baf1af519caf8d6839eeb6afb9b0"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.137421 7479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "4395dd876cdc4b21b6f7d8c30c28715e"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-pww0"
server_key: "814bd653f45686659a6480aefa5df6b9"
server_key_iv: "f104baf1af519caf8d6839eeb6afb9b0"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.143250 7479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:46:53.147267 7506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.148926 7479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:53.149120 7479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "4395dd876cdc4b21b6f7d8c30c28715e"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-pww0"
server_key: "814bd653f45686659a6480aefa5df6b9"
server_key_iv: "f104baf1af519caf8d6839eeb6afb9b0"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.149353 7479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:53.164265 7479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:53.169113 7479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:53.169430 7479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:53.170532 7479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:53.172178 7479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:53.172263 7479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.172385 7479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:53.172441 7479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.204411 7479 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:40511
I20260809 22:46:53.204478 7694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:40511 every 8 connection(s)
I20260809 22:46:53.205818 7479 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:53.213829 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 7479
I20260809 22:46:53.214043 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:53.214643 25120 external_mini_cluster.cc:1428] Setting key ab61fc79de7cac4fb04eaa84d077dc93
Aug 09 22:46:53 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315613, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:53.228641 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:53.208544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:45783 (local address 127.24.136.62:38915)
0809 22:46:53.208975 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:53.208979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:53.210049 (+ 1070us) server_negotiation.cc:407] Connection header received
0809 22:46:53.212025 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:53.212031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:53.212120 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:53.212232 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:53.213906 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.214993 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.216937 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.217569 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.217709 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:53.221804 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:53.221812 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:53.221816 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:53.221860 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:53.224872 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:53.225911 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:53.225916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:53.225918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:53.226065 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:53.226564 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:53.226567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:53.226569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:53.226846 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:53.226961 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:53.227930 (+ 969us) server_negotiation.cc:299] Negotiation successful
0809 22:46:53.228188 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":214,"threads_started":1}
I20260809 22:46:53.229595 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:53.209117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:45783)
0809 22:46:53.209873 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:53.209914 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:53.211708 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:53.212462 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:53.212471 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:53.213021 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:53.213665 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:53.213685 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.215164 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.215169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:53.216762 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:53.216780 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.217783 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.217788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:53.218009 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:53.218726 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:53.218751 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:53.221433 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:53.225064 (+ 3631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:53.225075 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:53.225096 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:53.225753 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:53.226175 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:53.226179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:53.226182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:53.226464 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:53.226965 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:53.226974 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:53.227496 (+ 522us) client_negotiation.cc:769] Sending connection context
0809 22:46:53.227810 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 22:46:53.228359 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":279,"threads_started":1}
I20260809 22:46:53.231169 7695 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38915
I20260809 22:46:53.231532 7695 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:53.232304 7695 heartbeater.cc:511] Master 127.24.136.62:38915 requested a full tablet report, sending...
I20260809 22:46:53.233906 7128 ts_manager.cc:194] Registered new tserver with Master: 4395dd876cdc4b21b6f7d8c30c28715e (127.24.136.2:40511)
I20260809 22:46:53.234793 7128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:45783
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:46:53.278393 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38915
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:53.631313 7708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:53.631672 7708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:53.631830 7708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:53.639346 7708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:53.639549 7708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:53.639716 7708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:53.650296 7708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:53.651681 7708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:53.653153 7708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:53.667490 7715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:53.667444 7714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:53.667593 7717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:53.668246 7708 server_base.cc:1034] running on GCE node
I20260809 22:46:53.668939 7708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:53.670012 7708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:53.671258 7708 hybrid_clock.cc:648] HybridClock initialized: now 1786315613671196 us; error 55 us; skew 500 ppm
I20260809 22:46:53.671696 7708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:53 dist-test-slave-pww0 krb5kdc[7044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315613, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:53.675172 7708 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:46:53.676837 7708 webserver.cc:460] Webserver started at http://127.24.136.3:42259/ using document root <none> and password file <none>
I20260809 22:46:53.677564 7708 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:53.677680 7708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:53.677961 7708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:53.681277 7708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "bb6ab8631dea461dae4294041f50b719"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-pww0"
server_key: "a0a0489b695be01bbaa299ad942d357e"
server_key_iv: "b0db656cc980b06e5619b6ff4c39da48"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.682101 7708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "bb6ab8631dea461dae4294041f50b719"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-pww0"
server_key: "a0a0489b695be01bbaa299ad942d357e"
server_key_iv: "b0db656cc980b06e5619b6ff4c39da48"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.685298 7472 heartbeater.cc:503] Master 127.24.136.62:38915 was elected leader, sending a full tablet report...
I20260809 22:46:53.688660 7708 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260809 22:46:53.692797 7728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.694394 7708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:46:53.694579 7708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "bb6ab8631dea461dae4294041f50b719"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-pww0"
server_key: "a0a0489b695be01bbaa299ad942d357e"
server_key_iv: "b0db656cc980b06e5619b6ff4c39da48"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.694795 7708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:53.710615 7708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:53.715101 7708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:53.715406 7708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:53.716517 7708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:53.717944 7708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:53.718027 7708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.718154 7708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:53.718219 7708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.750089 7708 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:33217
I20260809 22:46:53.750147 7920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:33217 every 8 connection(s)
I20260809 22:46:53.751576 7708 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:53.757495 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 7708
I20260809 22:46:53.757671 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:53.758307 25120 external_mini_cluster.cc:1428] Setting key 8a8a62b14371ca319088b387be071f54
Aug 09 22:46:53 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315613, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:46:53.772612 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:53.754345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:57787 (local address 127.24.136.62:38915)
0809 22:46:53.754862 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:53.754867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:53.755887 (+ 1020us) server_negotiation.cc:407] Connection header received
0809 22:46:53.757840 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:53.757847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:53.757959 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:53.758089 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:53.759941 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.760944 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.762777 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.763172 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.763285 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:53.766941 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:53.766947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:53.766950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:53.766989 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:53.769569 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:53.770359 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:53.770362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:53.770365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:53.770492 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:53.770867 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:53.770870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:53.770872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:53.771139 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:53.771259 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:53.772014 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 22:46:53.772253 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":263,"threads_started":1}
I20260809 22:46:53.773811 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:53.754939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:57787)
0809 22:46:53.755719 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:53.755765 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:53.757496 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:53.758295 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:53.758303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:53.758932 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:53.759761 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:53.759787 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.761143 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.761148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:53.762616 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:53.762630 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.763363 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.763368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:53.763637 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:53.764294 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:53.764320 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:53.766716 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:53.769732 (+ 3016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:53.769739 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:53.769756 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:53.770241 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:53.770570 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:53.770573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:53.770575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:53.770783 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:53.771234 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:53.771240 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:53.771640 (+ 400us) client_negotiation.cc:769] Sending connection context
0809 22:46:53.771879 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 22:46:53.772338 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":281,"threads_started":1}
I20260809 22:46:53.775398 7921 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38915
I20260809 22:46:53.775761 7921 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:53.776502 7921 heartbeater.cc:511] Master 127.24.136.62:38915 requested a full tablet report, sending...
I20260809 22:46:53.777983 7128 ts_manager.cc:194] Registered new tserver with Master: bb6ab8631dea461dae4294041f50b719 (127.24.136.3:33217)
I20260809 22:46:53.778841 7128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:57787
I20260809 22:46:53.787005 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab.
Entry for principal oryx/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab.
WARNING: no policy specified for oryx/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab.
Entry for principal oryx/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab.
WARNING: no policy specified for oryx/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab.
Entry for principal oryx/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab.
I20260809 22:46:53.958371 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 7226
Aug 09 22:46:53 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:46:53.975726 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:53.959683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.24.136.1:45569)
0809 22:46:53.960309 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:53.960318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:53.960346 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:46:53.960411 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:53.960419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:53.960645 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:53.960789 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:53.962226 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.963298 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.964657 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.965347 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.965476 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:53.969778 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:53.969794 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:53.969811 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:53.969858 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:53.972864 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:53.973662 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:53.973669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:53.973671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:53.973806 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:53.974108 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:53.974110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:53.974112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:53.974493 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:53.974740 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:53.975072 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 22:46:53.975309 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":291,"threads_started":1}
I20260809 22:46:53.977164 7458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:54.065439 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 7226
I20260809 22:46:54.085135 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 7479
Aug 09 22:46:54 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:46:54.102620 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.086206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.24.136.2:40511)
0809 22:46:54.086789 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.086796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.086824 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:46:54.086887 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.086893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.087112 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.087243 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:54.088642 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.089700 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.091073 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.091959 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.092086 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.096498 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:54.096513 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:54.096530 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:54.096574 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:54.099834 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.100607 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.100616 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.100618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.100761 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.101142 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.101145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.101147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.101509 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:54.101793 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.102064 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.102293 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":269,"threads_started":1}
I20260809 22:46:54.103987 7689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:54.188102 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 7479
I20260809 22:46:54.213421 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 7708
Aug 09 22:46:54 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:46:54.230737 7950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.214416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.24.136.3:33217)
0809 22:46:54.214957 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.214965 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.215000 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:46:54.215065 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.215073 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.215312 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.215494 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:54.216917 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.217947 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.219295 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.219986 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.220104 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.224709 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:54.224725 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:54.224742 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:54.224784 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:54.227849 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.228612 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.228621 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.228623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.228813 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.229193 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.229197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.229199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.229583 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:54.229878 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.230167 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.230413 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":238,"threads_started":1}
I20260809 22:46:54.232191 7913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:54.316140 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 7708
I20260809 22:46:54.342451 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 7069
I20260809 22:46:54.343853 7174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:54.448078 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 7069
I20260809 22:46:54.474018 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38915
--webserver_interface=127.24.136.62
--webserver_port=40141
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:54.831344 7956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:54.831772 7956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:54.831918 7956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:54.839551 7956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:54.839704 7956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:54.839797 7956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:54.839880 7956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:54.839943 7956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:54.850236 7956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=40141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:54.851770 7956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:54.853170 7956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:54.865563 7963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:54.865474 7968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:54.865525 7966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:54.866465 7956 server_base.cc:1034] running on GCE node
I20260809 22:46:54.867422 7956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:54.869045 7956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:54.870325 7956 hybrid_clock.cc:648] HybridClock initialized: now 1786315614870244 us; error 54 us; skew 500 ppm
I20260809 22:46:54.870710 7956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:54 dist-test-slave-pww0 krb5kdc[7044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315614, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:54.873986 7956 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:46:54.875684 7956 webserver.cc:460] Webserver started at http://127.24.136.62:40141/ using document root <none> and password file <none>
I20260809 22:46:54.876443 7956 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:54.876592 7956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:54.882834 7956 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:46:54.886081 7981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:54.887699 7956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:46:54.887887 7956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "75b3711de8af429c9c8e249518c9af72"
format_stamp: "Formatted at 2026-08-09 22:46:51 on dist-test-slave-pww0"
server_key: "96a224a94ff63d83636913cf9d23f8ef"
server_key_iv: "c7a35bd7e127c4593efa5214a17b58be"
server_key_version: "encryptionkey@0"
I20260809 22:46:54.888943 7956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:54.902086 7956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:54.906975 7956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:54.907267 7956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:54.926695 7956 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38915
I20260809 22:46:54.926717 8071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38915 every 8 connection(s)
I20260809 22:46:54.928396 7956 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:54.932883 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 7956
I20260809 22:46:54.933702 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:45569
--local_ip_for_outbound_sockets=127.24.136.1
--tserver_master_addrs=127.24.136.62:38915
--webserver_port=43735
--webserver_interface=127.24.136.1
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.136.1
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:54.936177 8072 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:46:54.939714 8072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: Bootstrap starting.
I20260809 22:46:54.964445 8072 log.cc:824] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:54.977262 8072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:46:54.977954 8072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: Bootstrap complete.
I20260809 22:46:54.991780 8072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } }
I20260809 22:46:54.993069 8072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b3711de8af429c9c8e249518c9af72, State: Initialized, Role: FOLLOWER
I20260809 22:46:54.993894 8072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } }
I20260809 22:46:54.994201 8072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:54.994333 8072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:54.994521 8072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:46:55.000550 8072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } }
I20260809 22:46:55.001286 8072 leader_election.cc:302] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b3711de8af429c9c8e249518c9af72; no voters:
I20260809 22:46:55.002070 8072 leader_election.cc:288] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:46:55.002230 8077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:46:55.003438 8077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [term 2 LEADER]: Becoming Leader. State: Replica: 75b3711de8af429c9c8e249518c9af72, State: Running, Role: LEADER
I20260809 22:46:55.004117 8077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } }
I20260809 22:46:55.004546 8072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:55.006803 8078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "75b3711de8af429c9c8e249518c9af72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } } }
I20260809 22:46:55.007234 8078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:55.006626 8079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b3711de8af429c9c8e249518c9af72. Latest consensus state: current_term: 2 leader_uuid: "75b3711de8af429c9c8e249518c9af72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b3711de8af429c9c8e249518c9af72" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38915 } } }
I20260809 22:46:55.008059 8079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:55.019196 8086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:55.026504 8086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:55.028524 8086 catalog_manager.cc:1259] Loaded cluster ID: 5ef5d603ae034142934c82920864fcd3
I20260809 22:46:55.028756 8086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:55.037784 8086 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:55.039950 8086 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 75b3711de8af429c9c8e249518c9af72: Loaded TSK: 0
I20260809 22:46:55.042006 8086 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:46:55.368820 8074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:55.369138 8074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:55.369254 8074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:55.376968 8074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:55.377101 8074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:55.377247 8074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:46:55.388154 8074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.1.keytab
--principal=oryx/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:45569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=43735
--tserver_master_addrs=127.24.136.62:38915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:55.389611 8074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:55.391052 8074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:55.407516 8113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:55.407539 8112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:55.408835 8116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:55.413335 8074 server_base.cc:1034] running on GCE node
I20260809 22:46:55.414068 8074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:55.415210 8074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:55.416464 8074 hybrid_clock.cc:648] HybridClock initialized: now 1786315615416401 us; error 56 us; skew 500 ppm
I20260809 22:46:55.416834 8074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:55 dist-test-slave-pww0 krb5kdc[7044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315615, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:55.420285 8074 init.cc:377] Logged in from keytab as oryx/127.24.136.1@KRBTEST.COM (short username oryx)
I20260809 22:46:55.422127 8074 webserver.cc:460] Webserver started at http://127.24.136.1:43735/ using document root <none> and password file <none>
I20260809 22:46:55.422924 8074 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:55.423064 8074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:55.429489 8074 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:46:55.432729 8131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:55.434338 8074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:46:55.434536 8074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "8defd5d77d7943ef8206ae87134fb444"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-pww0"
server_key: "2c08c31e23b39d2c6316190bcee187ea"
server_key_iv: "c414efafa216482455dc497f408b9c2e"
server_key_version: "encryptionkey@0"
I20260809 22:46:55.435678 8074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:55.455976 8074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:55.460619 8074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:55.460914 8074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:55.462057 8074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:55.463655 8074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:55.463771 8074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:55.463887 8074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:55.463958 8074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:55.497578 8074 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:45569
I20260809 22:46:55.497673 8313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:45569 every 8 connection(s)
I20260809 22:46:55.499119 8074 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:55.500626 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 8074
I20260809 22:46:55.501425 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:40511
--local_ip_for_outbound_sockets=127.24.136.2
--tserver_master_addrs=127.24.136.62:38915
--webserver_port=33545
--webserver_interface=127.24.136.2
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.136.2
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:46:55 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:55 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:46:55.860173 8321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:55.860499 8321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:55.860627 8321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:55.868532 8321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:55.868669 8321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:55.868829 8321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:46:55.879038 8321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.2.keytab
--principal=oryx/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:40511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=33545
--tserver_master_addrs=127.24.136.62:38915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:55.880589 8321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:55.881976 8321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:55.897022 8330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:55.896944 8331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:55.897581 8333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:55.898057 8321 server_base.cc:1034] running on GCE node
I20260809 22:46:55.898768 8321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:55.900100 8321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:55.901418 8321 hybrid_clock.cc:648] HybridClock initialized: now 1786315615901325 us; error 85 us; skew 500 ppm
I20260809 22:46:55.901786 8321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:55 dist-test-slave-pww0 krb5kdc[7044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315615, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:55.905802 8321 init.cc:377] Logged in from keytab as oryx/127.24.136.2@KRBTEST.COM (short username oryx)
I20260809 22:46:55.908306 8321 webserver.cc:460] Webserver started at http://127.24.136.2:33545/ using document root <none> and password file <none>
I20260809 22:46:55.909571 8321 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:55.909782 8321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:55.919341 8321 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.003s
I20260809 22:46:55.924242 8352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:55.926609 8321 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:46:55.926913 8321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "4395dd876cdc4b21b6f7d8c30c28715e"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-pww0"
server_key: "814bd653f45686659a6480aefa5df6b9"
server_key_iv: "f104baf1af519caf8d6839eeb6afb9b0"
server_key_version: "encryptionkey@0"
I20260809 22:46:55.928799 8321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:55.991767 8321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:55.996762 8321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:55.997053 8321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:55.998217 8321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:55.999761 8321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:55.999866 8321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:55.999974 8321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:56.000048 8321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.041517 8321 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:40511
I20260809 22:46:56.041561 8539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:40511 every 8 connection(s)
I20260809 22:46:56.043124 8321 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:56.044023 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 8321
I20260809 22:46:56.044819 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:33217
--local_ip_for_outbound_sockets=127.24.136.3
--tserver_master_addrs=127.24.136.62:38915
--webserver_port=42259
--webserver_interface=127.24.136.3
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.136.3
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:46:56 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:56 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:46:56.414288 8546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:56.414620 8546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:56.414806 8546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:56.422477 8546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:56.422621 8546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:56.422796 8546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:46:56.440889 8546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/krb5kdc/oryx_127.24.136.3.keytab
--principal=oryx/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:33217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=42259
--tserver_master_addrs=127.24.136.62:38915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:46:56.443130 8546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:56.445418 8546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:56.471740 8556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:56.473058 8557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:56.482466 8546 server_base.cc:1034] running on GCE node
W20260809 22:46:56.482785 8561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:56.484190 8546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:56.486009 8546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:56.487510 8546 hybrid_clock.cc:648] HybridClock initialized: now 1786315616487359 us; error 136 us; skew 500 ppm
I20260809 22:46:56.488123 8546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:56 dist-test-slave-pww0 krb5kdc[7044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315616, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:56.493884 8546 init.cc:377] Logged in from keytab as oryx/127.24.136.3@KRBTEST.COM (short username oryx)
I20260809 22:46:56.509913 8546 webserver.cc:460] Webserver started at http://127.24.136.3:42259/ using document root <none> and password file <none>
I20260809 22:46:56.511307 8546 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:56.511567 8546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:56.521533 8546 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.006s
I20260809 22:46:56.526770 8569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.529387 8546 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260809 22:46:56.529685 8546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "bb6ab8631dea461dae4294041f50b719"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-pww0"
server_key: "a0a0489b695be01bbaa299ad942d357e"
server_key_iv: "b0db656cc980b06e5619b6ff4c39da48"
server_key_version: "encryptionkey@0"
I20260809 22:46:56.531550 8546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:56.562534 8546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:56.570372 8546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:56.570777 8546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:56.572499 8546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:56.575049 8546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:56.575207 8546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.575350 8546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:56.575434 8546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.623348 8546 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:33217
I20260809 22:46:56.623411 8753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:33217 every 8 connection(s)
I20260809 22:46:56.624995 8546 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:56.626207 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 8546
I20260809 22:46:56.666330 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:56.627441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.24.136.62:38915)
0809 22:46:56.640058 (+ 12617us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:56.640065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:56.643188 (+ 3123us) server_negotiation.cc:407] Connection header received
0809 22:46:56.643399 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:56.643405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:56.643541 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:56.643712 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:56.646983 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.648619 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.650831 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.651561 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.651692 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.655718 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:56.655734 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:56.655762 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:56.655837 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:56.660329 (+ 4492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:56.661375 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:56.661386 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:56.661397 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:56.661676 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:56.662192 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:56.662197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:56.662200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:56.662892 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:56.663208 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:56.664707 (+ 1499us) server_negotiation.cc:299] Negotiation successful
0809 22:46:56.665242 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":12451,"thread_start_us":16267,"threads_started":1}
Aug 09 22:46:56 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:56 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:46:56.779138 8316 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.263s user 0.004s sys 0.009s
W20260809 22:46:56.781203 8318 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:55.502712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:47989 (local address 127.24.136.62:38915)
0809 22:46:55.506690 (+ 3978us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:55.506708 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:55.506759 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:46:55.507157 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:55.507176 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:55.507681 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:55.508115 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:55.510344 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:55.511756 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:55.513977 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:55.515111 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:55.515311 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.780784 (+1265473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:47989: BlockingRecv error: recv got EOF from 127.24.136.1:47989 (error 108)
Metrics: {"server-negotiator.queue_time_us":3699,"thread_start_us":3452,"threads_started":1}
I20260809 22:46:56.781824 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:55.502754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:47989)
0809 22:46:55.503673 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:55.503723 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:55.505817 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:55.508381 (+ 2564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:55.508394 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:55.509223 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:55.510033 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:55.510062 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:55.511991 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:55.511997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:55.513787 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:55.513804 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:55.515389 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:55.515395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:55.515665 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:55.516351 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:55.516382 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:56.780353 (+1263971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":365,"threads_started":1}
W20260809 22:46:56.783282 8314 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:38915 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.136.62:38915: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Aug 09 22:46:56 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:56 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:46:56.799099 8318 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:56.784350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:59927 (local address 127.24.136.62:38915)
0809 22:46:56.784565 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:56.784573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:56.784739 (+ 166us) server_negotiation.cc:407] Connection header received
0809 22:46:56.784978 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:56.784984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:56.785085 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:56.785215 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:56.786304 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.787678 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.790043 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.790848 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.791004 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.798779 (+ 7775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:59927: BlockingRecv error: recv got EOF from 127.24.136.1:59927 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:46:56.800163 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:56.784272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:59927)
0809 22:46:56.784625 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:56.784649 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:56.784809 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:56.785257 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:56.785262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:56.785826 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:56.786100 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.786125 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.787936 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.787944 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.789804 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.789828 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.791366 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.791376 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.791641 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.792530 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:56.792566 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:56.799780 (+ 7214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206}
Aug 09 22:46:56 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:56 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:46:56.816311 8318 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:56.801511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:39503 (local address 127.24.136.62:38915)
0809 22:46:56.801703 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:56.801710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:56.802020 (+ 310us) server_negotiation.cc:407] Connection header received
0809 22:46:56.802339 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:56.802346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:56.802442 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:56.802523 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:56.804369 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.805689 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.807947 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.808742 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.808901 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.815947 (+ 7046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:39503: BlockingRecv error: recv got EOF from 127.24.136.1:39503 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:46:56.817875 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:56.801344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:39503)
0809 22:46:56.801911 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:56.801936 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:56.802161 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:56.802804 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:56.802809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:56.803864 (+ 1055us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:56.804144 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.804158 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.805870 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.805877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.807751 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.807772 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.808934 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.808941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.809199 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.810056 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:56.810084 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:56.815530 (+ 5446us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423}
W20260809 22:46:56.818861 8314 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:46:57.225994 8544 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.154s user 0.010s sys 0.011s
W20260809 22:46:57.227761 8545 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:56.048126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:49487 (local address 127.24.136.62:38915)
0809 22:46:56.059965 (+ 11839us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:56.059970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:56.060666 (+ 696us) server_negotiation.cc:407] Connection header received
0809 22:46:56.063125 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:56.063133 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:56.063255 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:56.063402 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:56.065602 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.066948 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.069393 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.070025 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.070156 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.227433 (+1157277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:49487: BlockingRecv error: recv got EOF from 127.24.136.2:49487 (error 108)
Metrics: {"server-negotiator.queue_time_us":11716,"thread_start_us":11577,"threads_started":1}
I20260809 22:46:57.228703 8544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:56.047318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:49487)
0809 22:46:56.060477 (+ 13159us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:56.060520 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:56.062768 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:56.063761 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:56.063773 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:56.064504 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:56.065357 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.065391 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.067156 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.067161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.069176 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.069192 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.070351 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.070360 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.070747 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.071722 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:56.071758 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:57.227223 (+1155465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":12850,"thread_start_us":14354,"threads_started":1}
W20260809 22:46:57.229744 8540 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:38915 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.136.62:38915: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:46:57.245136 8545 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:57.231001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:56969 (local address 127.24.136.62:38915)
0809 22:46:57.231160 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.231164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.231680 (+ 516us) server_negotiation.cc:407] Connection header received
0809 22:46:57.231874 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.231878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.231960 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.232050 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.233178 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.233941 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.235324 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.236122 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.236289 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.244816 (+ 8527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:56969: BlockingRecv error: recv got EOF from 127.24.136.2:56969 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 22:46:57.245800 8544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.230510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:56969)
0809 22:46:57.231594 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.231615 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.231756 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.232259 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.232263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.232749 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:57.232991 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.233006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.234119 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.234123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.235109 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.235121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.236343 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.236351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.236511 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.240113 (+ 3602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:57.240136 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:57.245459 (+ 5323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":917}
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:57.262315 8544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.246623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:59627)
0809 22:46:57.246819 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.246843 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.246994 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.247385 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.247391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.247942 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:57.248186 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.248202 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.249993 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.250000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.251839 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.251861 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.253056 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.253064 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.253301 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.254073 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:57.254102 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:57.261941 (+ 7839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 22:46:57.262590 8545 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:57.246899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:59627 (local address 127.24.136.62:38915)
0809 22:46:57.247073 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.247078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.247094 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:46:57.247143 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.247146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.247229 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.247316 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.248410 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.249715 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.252140 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.252839 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.252996 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.262282 (+ 9286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:59627: BlockingRecv error: recv got EOF from 127.24.136.2:59627 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 22:46:57.263094 8540 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:57.829793 8762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.819529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:40657)
0809 22:46:57.820177 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.820199 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.820326 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.820705 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.820709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.821146 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:57.821303 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.821312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.822711 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.822714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.823758 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.823774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.824785 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.824790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.824948 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.825592 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:57.825612 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:57.829428 (+ 3816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":319,"threads_started":1}
W20260809 22:46:57.830020 8763 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:57.819578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:40657 (local address 127.24.136.62:38915)
0809 22:46:57.820377 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.820384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.820403 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:46:57.820472 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.820477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.820560 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.820671 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.821441 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.822581 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.823900 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.824591 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.824727 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.829752 (+ 5025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:40657: BlockingRecv error: recv got EOF from 127.24.136.1:40657 (error 108)
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":571,"threads_started":1}
W20260809 22:46:57.870011 8759 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.195s user 0.001s sys 0.016s
W20260809 22:46:57.871269 8758 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:56.631217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:60117 (local address 127.24.136.62:38915)
0809 22:46:56.644086 (+ 12869us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:56.644092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:56.644116 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:46:56.644177 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:56.644184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:56.644272 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:56.644394 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:56.648405 (+ 4011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.651569 (+ 3164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.672399 (+ 20830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.673167 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.673355 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.871005 (+1197650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:60117: BlockingRecv error: recv got EOF from 127.24.136.3:60117 (error 108)
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":12705,"thread_start_us":8512,"threads_started":1}
I20260809 22:46:57.871784 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:56.629916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:60117)
0809 22:46:56.636302 (+ 6386us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:56.636386 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:56.639102 (+ 2716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:56.646038 (+ 6936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:56.646058 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:56.647160 (+ 1102us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:56.648151 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.648182 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.651861 (+ 3679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.651869 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.672172 (+ 20303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.672202 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.674063 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.674074 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.674431 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.675288 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:56.675315 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:57.870799 (+1195484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":5887,"thread_start_us":6921,"threads_started":1}
W20260809 22:46:57.872754 8754 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:38915 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.136.62:38915: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:57.882705 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.873399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:59863)
0809 22:46:57.873598 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.873636 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.873737 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.874056 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.874059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.874400 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:57.874570 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.874578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.875776 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.875787 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.876829 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.876844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.877604 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.877609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.877739 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.878347 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:57.878368 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:57.882466 (+ 4098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 22:46:57.882874 8758 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:57.873538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:59863 (local address 127.24.136.62:38915)
0809 22:46:57.873794 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.873798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.873815 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:46:57.873865 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.873868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.873933 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.874012 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.874718 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.875616 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.876977 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.877409 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.877509 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.882673 (+ 5164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:59863: BlockingRecv error: recv got EOF from 127.24.136.3:59863 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:57 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:57.891424 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.883395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:43643)
0809 22:46:57.883557 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.883577 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.883680 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.883935 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.883938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.884232 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:57.884403 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.884411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.885462 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.885466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.886472 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.886485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.887170 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.887174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.887315 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.887905 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:57.887924 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:57.891213 (+ 3289us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 22:46:57.891746 8758 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:57.883504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:43643 (local address 127.24.136.62:38915)
0809 22:46:57.883666 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.883670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.883686 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:46:57.883761 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.883764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.883822 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.883900 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.884544 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.885328 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.886616 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.887046 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.887149 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.891443 (+ 4294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:43643: BlockingRecv error: recv got EOF from 127.24.136.3:43643 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 22:46:57.891789 8754 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 22:46:58 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:58 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:58.276306 8764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:58.263811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:50429)
0809 22:46:58.264340 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:58.264358 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:58.264479 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:58.264777 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:58.264780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:58.265231 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:58.265391 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:58.265400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.266394 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.266399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:58.267417 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:58.267432 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.268183 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.268188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:58.268319 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:58.271819 (+ 3500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:58.271840 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:58.275968 (+ 4128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":301,"threads_started":1}
W20260809 22:46:58.276472 8758 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:58.263898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:50429 (local address 127.24.136.62:38915)
0809 22:46:58.264162 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:58.264169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:58.264435 (+ 266us) server_negotiation.cc:407] Connection header received
0809 22:46:58.264572 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:58.264576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:58.264659 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:58.264750 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:58.265522 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.266254 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.267599 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.268018 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.268114 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:58.276224 (+ 8110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:50429: BlockingRecv error: recv got EOF from 127.24.136.2:50429 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 09 22:46:58 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:58 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:58.841297 8765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:58.831280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:49675)
0809 22:46:58.831940 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:58.831966 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:58.832125 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:58.832524 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:58.832528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:58.833009 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:58.833252 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:58.833265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.834543 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.834548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:58.835516 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:58.835529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.836382 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.836389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:58.836588 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:58.837183 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:58.837201 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:58.840937 (+ 3736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":355,"threads_started":1}
W20260809 22:46:58.841457 8767 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:58.831326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:49675 (local address 127.24.136.62:38915)
0809 22:46:58.832201 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:58.832207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:58.832224 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:46:58.832281 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:58.832285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:58.832352 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:58.832444 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:58.833423 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.834367 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.835688 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.836217 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.836356 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:58.841203 (+ 4847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:49675: BlockingRecv error: recv got EOF from 127.24.136.1:49675 (error 108)
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":641,"threads_started":1}
Aug 09 22:46:58 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:58 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:58.901903 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:58.892495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:51843)
0809 22:46:58.893101 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:58.893122 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:58.893277 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:58.893658 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:58.893663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:58.894122 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:58.894292 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:58.894302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.895449 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.895483 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:58.896595 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:58.896610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.897346 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.897352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:58.897515 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:58.898097 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:58.898114 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:58.901549 (+ 3435us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":350,"threads_started":1}
W20260809 22:46:58.902038 8767 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:58.892654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:51843 (local address 127.24.136.62:38915)
0809 22:46:58.892923 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:58.892929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:58.893211 (+ 282us) server_negotiation.cc:407] Connection header received
0809 22:46:58.893404 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:58.893409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:58.893486 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:58.893571 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:58.894446 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.895301 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.896735 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:58.897152 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:58.897262 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:58.901819 (+ 4557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:51843: BlockingRecv error: recv got EOF from 127.24.136.3:51843 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 22:46:59 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:59 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:59.286929 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.277377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:55927)
0809 22:46:59.277898 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.277915 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:59.278089 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:59.278590 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:59.278595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:59.279022 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:59.279224 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:59.279233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.280388 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.280394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:59.281406 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:59.281421 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.282165 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.282170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:59.282319 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:59.282939 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:59.282958 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:59.286672 (+ 3714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":302,"threads_started":1}
W20260809 22:46:59.287190 8767 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:59.277507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:55927 (local address 127.24.136.62:38915)
0809 22:46:59.277682 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:59.277686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:59.278002 (+ 316us) server_negotiation.cc:407] Connection header received
0809 22:46:59.278367 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:59.278372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:59.278443 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:59.278525 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:59.279405 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.280208 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.281562 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.281996 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.282097 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:59.286899 (+ 4802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:55927: BlockingRecv error: recv got EOF from 127.24.136.2:55927 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 22:46:59 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:59 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:59.852231 8772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.842366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:39451)
0809 22:46:59.842986 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.843009 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:59.843133 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:59.843541 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:59.843545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:59.843931 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:59.844139 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:59.844149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.845199 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.845203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:59.846146 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:59.846159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.846890 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.846894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:59.847021 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:59.847882 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:59.847901 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:59.851962 (+ 4061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":287,"threads_started":1}
W20260809 22:46:59.852504 8775 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:59.842625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:39451 (local address 127.24.136.62:38915)
0809 22:46:59.843086 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:59.843091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:59.843108 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:46:59.843246 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:59.843250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:59.843326 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:59.843421 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:59.844304 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.845058 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.846283 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.846757 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.846853 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:59.852216 (+ 5363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:39451: BlockingRecv error: recv got EOF from 127.24.136.1:39451 (error 108)
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":246,"threads_started":1}
Aug 09 22:46:59 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:46:59 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:46:59.912405 8779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.902921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:41523)
0809 22:46:59.903503 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.903528 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:59.903685 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:59.904042 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:59.904048 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:59.904481 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:59.904672 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:59.904682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.905810 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.905815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:59.906819 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:59.906835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.907660 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.907665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:59.907822 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:59.908509 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:59.908529 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:59.912146 (+ 3617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":329,"threads_started":1}
W20260809 22:46:59.912873 8775 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:46:59.903048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:41523 (local address 127.24.136.62:38915)
0809 22:46:59.903269 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:59.903276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:59.903636 (+ 360us) server_negotiation.cc:407] Connection header received
0809 22:46:59.903831 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:59.903835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:59.903904 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:59.903984 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:59.904841 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.905633 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.906984 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:59.907450 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:59.907603 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:59.912426 (+ 4823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:41523: BlockingRecv error: recv got EOF from 127.24.136.3:41523 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 22:47:00 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:00 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:00.297952 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.288030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:34501)
0809 22:47:00.288610 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:00.288629 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:00.288759 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:00.289067 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:00.289070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:00.289449 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:00.289657 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.289666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.290764 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.290769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.291752 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.291767 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.292643 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.292648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.292788 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.293422 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:00.293440 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:00.297663 (+ 4223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":286,"threads_started":1}
W20260809 22:47:00.298074 8775 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:00.288194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:34501 (local address 127.24.136.62:38915)
0809 22:47:00.288394 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:00.288399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:00.288703 (+ 304us) server_negotiation.cc:407] Connection header received
0809 22:47:00.288854 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:00.288857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:00.288939 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:00.289032 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:00.289814 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.290590 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.291928 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.292452 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.292570 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.297850 (+ 5280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:34501: BlockingRecv error: recv got EOF from 127.24.136.2:34501 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 22:47:00 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:00 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:00.862944 8783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.853457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:52043)
0809 22:47:00.854117 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:00.854144 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:00.854302 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:00.854746 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:00.854752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:00.855258 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:00.855505 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.855520 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.856614 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.856620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.857629 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.857643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.858348 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.858355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.858544 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.859152 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:00.859170 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:00.862596 (+ 3426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":308,"threads_started":1}
W20260809 22:47:00.863174 8787 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:00.853595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:52043 (local address 127.24.136.62:38915)
0809 22:47:00.854411 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:00.854417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:00.854441 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:47:00.854499 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:00.854502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:00.854564 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:00.854675 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:00.855675 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.856473 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.857795 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.858202 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.858314 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.862925 (+ 4611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:52043: BlockingRecv error: recv got EOF from 127.24.136.1:52043 (error 108)
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":522,"threads_started":1}
Aug 09 22:47:00 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:00 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:00.923171 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.913563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:60819)
0809 22:47:00.914187 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:00.914212 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:00.914366 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:00.914728 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:00.914733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:00.915137 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:00.915353 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.915364 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.916491 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.916495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.917572 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.917592 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.918394 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.918399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.918581 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.919302 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:00.919322 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:00.922918 (+ 3596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":338,"threads_started":1}
W20260809 22:47:00.923394 8787 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:00.913735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:60819 (local address 127.24.136.62:38915)
0809 22:47:00.914034 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:00.914040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:00.914304 (+ 264us) server_negotiation.cc:407] Connection header received
0809 22:47:00.914499 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:00.914503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:00.914580 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:00.914670 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:00.915549 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.916305 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.917787 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.918222 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.918359 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.923170 (+ 4811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:60819: BlockingRecv error: recv got EOF from 127.24.136.3:60819 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 09 22:47:01 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:01 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:01.308363 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:01.298920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:55251)
0809 22:47:01.299427 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:01.299443 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:01.299602 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:01.299948 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:01.299953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:01.300396 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:01.300566 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:01.300576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.301626 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.301632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:01.302646 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:01.302658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.303403 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.303409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:01.303613 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:01.304275 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:01.304293 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:01.307993 (+ 3700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":279,"threads_started":1}
W20260809 22:47:01.308533 8787 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:01.298992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:55251 (local address 127.24.136.62:38915)
0809 22:47:01.299214 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:01.299220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:01.299546 (+ 326us) server_negotiation.cc:407] Connection header received
0809 22:47:01.299718 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:01.299725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:01.299811 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:01.299896 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:01.300715 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.301493 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.302798 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.303255 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.303368 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:01.308266 (+ 4898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:55251: BlockingRecv error: recv got EOF from 127.24.136.2:55251 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 22:47:01 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:01 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:01.874066 8793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:01.864187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:42119)
0809 22:47:01.865028 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:01.865045 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:01.865174 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:01.865593 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:01.865597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:01.866121 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:01.866289 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:01.866299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.867499 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.867503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:01.868488 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:01.868501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.869306 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.869311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:01.869452 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:01.870119 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:01.870141 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:01.873829 (+ 3688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":577,"threads_started":1}
W20260809 22:47:01.874369 8794 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:01.864415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:42119 (local address 127.24.136.62:38915)
0809 22:47:01.864958 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:01.864964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:01.865120 (+ 156us) server_negotiation.cc:407] Connection header received
0809 22:47:01.865302 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:01.865309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:01.865413 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:01.865542 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:01.866429 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.867306 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.868644 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.869129 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.869249 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:01.874102 (+ 4853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:42119: BlockingRecv error: recv got EOF from 127.24.136.1:42119 (error 108)
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":276,"threads_started":1}
Aug 09 22:47:01 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:01 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:01.935452 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:01.924351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:50709)
0809 22:47:01.925031 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:01.925069 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:01.925224 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:01.925619 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:01.925623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:01.926048 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:01.926242 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:01.926255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.927713 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.927720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:01.929535 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:01.929555 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.930543 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.930550 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:01.930699 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:01.931383 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:01.931404 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:01.935193 (+ 3789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":314,"threads_started":1}
W20260809 22:47:01.935868 8794 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:01.924446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:50709 (local address 127.24.136.62:38915)
0809 22:47:01.924636 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:01.924641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:01.925184 (+ 543us) server_negotiation.cc:407] Connection header received
0809 22:47:01.925336 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:01.925340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:01.925408 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:01.925499 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:01.926425 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.927540 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.929692 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:01.930373 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:01.930493 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:01.935490 (+ 4997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:50709: BlockingRecv error: recv got EOF from 127.24.136.3:50709 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 22:47:02 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:02 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:02.318476 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:02.309453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:38349)
0809 22:47:02.309953 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:02.309970 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:02.310091 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:02.310442 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:02.310446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:02.310895 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:02.311069 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:02.311078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.312161 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.312166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:02.313125 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:02.313138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.313866 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.313871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:02.314080 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:02.314713 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:02.314732 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:02.318242 (+ 3510us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":267,"threads_started":1}
W20260809 22:47:02.318737 8794 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:02.309627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:38349 (local address 127.24.136.62:38915)
0809 22:47:02.309797 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:02.309802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:02.310076 (+ 274us) server_negotiation.cc:407] Connection header received
0809 22:47:02.310208 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:02.310212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:02.310290 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:02.310376 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:02.311198 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.312021 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.313271 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.313698 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.313796 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:02.318486 (+ 4690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:38349: BlockingRecv error: recv got EOF from 127.24.136.2:38349 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 22:47:02 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:02 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:02.884984 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:02.875482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:56671)
0809 22:47:02.876014 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:02.876032 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:02.876152 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:02.876732 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:02.876736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:02.877139 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:02.877315 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:02.877324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.878656 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.878661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:02.879800 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:02.879817 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.880510 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.880514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:02.880635 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:02.881219 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:02.881236 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:02.884747 (+ 3511us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":274,"threads_started":1}
W20260809 22:47:02.885217 8805 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:02.875600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:56671 (local address 127.24.136.62:38915)
0809 22:47:02.876407 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:02.876412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:02.876429 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:47:02.876485 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:02.876489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:02.876555 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:02.876653 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:02.877459 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.878489 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.879950 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.880378 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.880476 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:02.884958 (+ 4482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:56671: BlockingRecv error: recv got EOF from 127.24.136.1:56671 (error 108)
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":577,"threads_started":1}
Aug 09 22:47:02 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:02 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:02.946352 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:02.936603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:40325)
0809 22:47:02.937117 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:02.937139 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:02.937262 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:02.937584 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:02.937588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:02.938021 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:02.938210 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:02.938222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.939385 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.939392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:02.940584 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:02.940603 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.941534 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.941541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:02.941770 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:02.942440 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:02.942460 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:02.946096 (+ 3636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":252,"threads_started":1}
W20260809 22:47:02.946609 8805 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:02.936715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:40325 (local address 127.24.136.62:38915)
0809 22:47:02.936893 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:02.936898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:02.937214 (+ 316us) server_negotiation.cc:407] Connection header received
0809 22:47:02.937361 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:02.937365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:02.937434 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:02.937555 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:02.938351 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.939148 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.940748 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:02.941351 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:02.941461 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:02.946382 (+ 4921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:40325: BlockingRecv error: recv got EOF from 127.24.136.3:40325 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 22:47:03 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:03 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:03.333864 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:03.319707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:43767)
0809 22:47:03.320220 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:03.320238 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:03.320407 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:03.320849 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:03.320853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:03.321295 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:03.321482 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:03.321491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.322807 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.322814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:03.323803 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:03.323820 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.324618 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.324624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:03.324802 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:03.329539 (+ 4737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:03.329568 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:03.333602 (+ 4034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":276,"threads_started":1}
W20260809 22:47:03.334249 8805 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:03.319875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:43767 (local address 127.24.136.62:38915)
0809 22:47:03.320126 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:03.320133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:03.320373 (+ 240us) server_negotiation.cc:407] Connection header received
0809 22:47:03.320595 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:03.320600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:03.320679 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:03.320781 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:03.321681 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.322622 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.323972 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.324438 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.324559 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:03.333928 (+ 9369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:43767: BlockingRecv error: recv got EOF from 127.24.136.2:43767 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 22:47:03 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:03 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:03.895430 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:03.885931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:54395)
0809 22:47:03.886544 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:03.886561 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:03.886682 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:03.887197 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:03.887201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:03.887765 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:03.887952 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:03.887962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.889042 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.889047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:03.890007 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:03.890020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.890799 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.890805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:03.890945 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:03.891592 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:03.891612 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:03.895158 (+ 3546us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":264,"threads_started":1}
W20260809 22:47:03.895841 8811 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:03.886077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:54395 (local address 127.24.136.62:38915)
0809 22:47:03.886878 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:03.886882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:03.886898 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:47:03.886947 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:03.886951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:03.887020 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:03.887112 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:03.888098 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.888887 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.890153 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.890621 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.890718 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:03.895520 (+ 4802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:54395: BlockingRecv error: recv got EOF from 127.24.136.1:54395 (error 108)
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":561,"threads_started":1}
Aug 09 22:47:03 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:03 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:03.956826 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:03.947526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:39031)
0809 22:47:03.948094 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:03.948124 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:03.948261 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:03.948693 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:03.948697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:03.949133 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:03.949318 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:03.949328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.950428 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.950433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:03.951423 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:03.951437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.952325 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.952330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:03.952475 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:03.953053 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:03.953070 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:03.956588 (+ 3518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":302,"threads_started":1}
W20260809 22:47:03.957001 8811 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:03.947643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:39031 (local address 127.24.136.62:38915)
0809 22:47:03.947830 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:03.947835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:03.948229 (+ 394us) server_negotiation.cc:407] Connection header received
0809 22:47:03.948419 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:03.948423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:03.948493 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:03.948597 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:03.949479 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.950247 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.951658 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:03.952118 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:03.952230 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:03.956810 (+ 4580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:39031: BlockingRecv error: recv got EOF from 127.24.136.3:39031 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 22:47:04 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:04 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:04.344925 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:04.335083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:39299)
0809 22:47:04.335771 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:04.335788 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:04.335923 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:04.336301 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:04.336305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:04.336749 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:04.336976 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:04.336986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.338251 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.338256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:04.339435 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:04.339478 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.340285 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.340292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:04.340470 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:04.341073 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:04.341091 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:04.344670 (+ 3579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":307,"threads_started":1}
W20260809 22:47:04.345357 8811 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:04.335221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:39299 (local address 127.24.136.62:38915)
0809 22:47:04.335528 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:04.335533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:04.335881 (+ 348us) server_negotiation.cc:407] Connection header received
0809 22:47:04.336078 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:04.336082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:04.336155 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:04.336237 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:04.337169 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.338081 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.339606 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.340113 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.340240 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:04.344899 (+ 4659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:39299: BlockingRecv error: recv got EOF from 127.24.136.2:39299 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 09 22:47:04 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:04 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:04.906455 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:04.896580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:35575)
0809 22:47:04.897182 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:04.897199 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:04.897330 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:04.897691 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:04.897695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:04.898137 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:04.898321 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:04.898330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.899422 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.899427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:04.900433 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:04.900448 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.901215 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.901220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:04.901338 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:04.902007 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:04.902032 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:04.906088 (+ 4056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":276,"threads_started":1}
W20260809 22:47:04.906697 8816 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:04.896673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:35575 (local address 127.24.136.62:38915)
0809 22:47:04.897212 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:04.897217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:04.897273 (+ 56us) server_negotiation.cc:407] Connection header received
0809 22:47:04.897446 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:04.897450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:04.897517 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:04.897605 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:04.898465 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.899266 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.900587 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.901055 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.901154 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:04.906429 (+ 5275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:35575: BlockingRecv error: recv got EOF from 127.24.136.1:35575 (error 108)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":308,"threads_started":1}
Aug 09 22:47:04 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:04 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:04.967515 8817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:04.957845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:45079)
0809 22:47:04.958511 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:04.958533 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:04.958657 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:04.958974 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:04.958977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:04.959403 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:04.959617 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:04.959626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.960681 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.960686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:04.961655 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:04.961667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.962437 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.962443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:04.962603 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:04.963303 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:04.963323 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:04.967140 (+ 3817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":322,"threads_started":1}
W20260809 22:47:04.967662 8816 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:04.958043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:45079 (local address 127.24.136.62:38915)
0809 22:47:04.958248 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:04.958253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:04.958642 (+ 389us) server_negotiation.cc:407] Connection header received
0809 22:47:04.958769 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:04.958773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:04.958848 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:04.958955 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:04.959783 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.960547 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.961802 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:04.962247 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:04.962349 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:04.967395 (+ 5046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:45079: BlockingRecv error: recv got EOF from 127.24.136.3:45079 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 22:47:05 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:05 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:05.356493 8818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:05.346152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:40225)
0809 22:47:05.346767 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:05.346785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:05.346922 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:05.347318 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:05.347322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:05.347854 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:05.348068 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:05.348088 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.349537 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.349542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:05.350528 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:05.350542 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.351428 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.351434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:05.351645 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:05.352332 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:05.352352 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:05.356215 (+ 3863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":295,"threads_started":1}
W20260809 22:47:05.356735 8816 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:05.346410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:40225 (local address 127.24.136.62:38915)
0809 22:47:05.346594 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:05.346598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:05.346890 (+ 292us) server_negotiation.cc:407] Connection header received
0809 22:47:05.347058 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:05.347062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:05.347157 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:05.347251 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:05.348247 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.349371 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.350717 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.351214 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.351334 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:05.356499 (+ 5165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:40225: BlockingRecv error: recv got EOF from 127.24.136.2:40225 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 22:47:05 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:05 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:05.917742 8823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:05.907676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:45539)
0809 22:47:05.908322 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:05.908345 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:05.908483 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:05.908966 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:05.908971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:05.909449 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:05.909621 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:05.909631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.910784 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.910788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:05.911859 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:05.911874 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.912709 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.912715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:05.912861 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:05.913498 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:05.913518 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:05.917490 (+ 3972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":269,"threads_started":1}
W20260809 22:47:05.918082 8824 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:05.907709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:45539 (local address 127.24.136.62:38915)
0809 22:47:05.908473 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:05.908478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:05.908495 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:47:05.908662 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:05.908668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:05.908756 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:05.908883 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:05.909776 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.910548 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.912031 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.912518 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.912635 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:05.917788 (+ 5153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:45539: BlockingRecv error: recv got EOF from 127.24.136.1:45539 (error 108)
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":513,"threads_started":1}
Aug 09 22:47:05 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:05 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:05.978729 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:05.968628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:50741)
0809 22:47:05.969296 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:05.969319 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:05.969489 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:05.969991 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:05.969995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:05.970647 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:05.970875 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:05.970887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.972342 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.972346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:05.973357 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:05.973373 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.974118 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.974123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:05.974253 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:05.974891 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:05.974910 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:05.978468 (+ 3558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":363,"threads_started":1}
W20260809 22:47:05.978928 8824 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:05.968860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:50741 (local address 127.24.136.62:38915)
0809 22:47:05.969146 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:05.969153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:05.969429 (+ 276us) server_negotiation.cc:407] Connection header received
0809 22:47:05.969688 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:05.969692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:05.969788 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:05.969915 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:05.971068 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.972178 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.973522 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:05.973946 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:05.974048 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:05.978708 (+ 4660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:50741: BlockingRecv error: recv got EOF from 127.24.136.3:50741 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 09 22:47:06 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:06 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:06.366968 8828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:06.357677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:45369)
0809 22:47:06.358222 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:06.358238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:06.358362 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:06.358665 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:06.358668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:06.359096 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:06.359247 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:06.359255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.360336 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.360341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:06.361316 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:06.361330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.362065 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.362071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:06.362201 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:06.362804 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:06.362822 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:06.366735 (+ 3913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":284,"threads_started":1}
W20260809 22:47:06.367239 8824 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:06.357760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:45369 (local address 127.24.136.62:38915)
0809 22:47:06.357946 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:06.357951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:06.358315 (+ 364us) server_negotiation.cc:407] Connection header received
0809 22:47:06.358459 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:06.358463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:06.358543 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:06.358634 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:06.359425 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.360198 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.361474 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.361921 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.362021 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:06.366962 (+ 4941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:45369: BlockingRecv error: recv got EOF from 127.24.136.2:45369 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 22:47:06 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:06 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:06.931059 8829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:06.918942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:52585)
0809 22:47:06.919570 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:06.919593 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:06.919758 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:06.920145 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:06.920150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:06.920616 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:06.920808 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:06.920818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.922480 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.922487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:06.924197 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:06.924220 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.925028 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.925034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:06.925230 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:06.925927 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:06.925953 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:06.930718 (+ 4765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":298,"threads_started":1}
W20260809 22:47:06.931332 8831 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:06.919048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:52585 (local address 127.24.136.62:38915)
0809 22:47:06.919843 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:06.919850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:06.919869 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:47:06.919924 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:06.919928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:06.919988 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:06.920081 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:06.921011 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.922248 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.924378 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.924859 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.924964 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:06.931032 (+ 6068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:52585: BlockingRecv error: recv got EOF from 127.24.136.1:52585 (error 108)
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":542,"threads_started":1}
Aug 09 22:47:06 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:06 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:06.990302 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:06.979945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:50347)
0809 22:47:06.980496 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:06.980517 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:06.980668 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:06.981001 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:06.981006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:06.981449 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:06.981663 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:06.981674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.982758 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.982763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:06.983972 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:06.983987 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.984860 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.984866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:06.985066 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:06.985722 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:06.985748 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:06.990038 (+ 4290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":305,"threads_started":1}
W20260809 22:47:06.990882 8831 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:06.980007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:50347 (local address 127.24.136.62:38915)
0809 22:47:06.980203 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:06.980211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:06.980619 (+ 408us) server_negotiation.cc:407] Connection header received
0809 22:47:06.980795 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:06.980800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:06.980868 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:06.980951 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:06.981822 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.982615 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.984120 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:06.984693 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:06.985129 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:06.990329 (+ 5200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:50347: BlockingRecv error: recv got EOF from 127.24.136.3:50347 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 22:47:07 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:07 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:07.378175 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:07.368222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:57363)
0809 22:47:07.368930 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:07.368947 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:07.369080 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:07.369458 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:07.369462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:07.369862 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:07.370062 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:07.370072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.371158 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.371162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:07.372258 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:07.372273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.373148 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.373153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:07.373300 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:07.374089 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:07.374115 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:07.377913 (+ 3798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":466,"threads_started":1}
W20260809 22:47:07.378420 8831 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:07.368329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:57363 (local address 127.24.136.62:38915)
0809 22:47:07.368581 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:07.368586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:07.369056 (+ 470us) server_negotiation.cc:407] Connection header received
0809 22:47:07.369177 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:07.369182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:07.369282 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:07.369382 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:07.370238 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.371022 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.372465 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.372944 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.373089 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:07.378150 (+ 5061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:57363: BlockingRecv error: recv got EOF from 127.24.136.2:57363 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 22:47:07 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:07 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:07.943054 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:07.932286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:51785)
0809 22:47:07.933201 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:07.933223 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:07.933376 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:07.933717 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:07.933723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:07.934282 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:07.934509 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:07.934521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.935727 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.935733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:07.937044 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:07.937061 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.937864 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.937869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:07.938010 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:07.938717 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:07.938738 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:07.942740 (+ 4002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":662,"threads_started":1}
W20260809 22:47:07.943380 8836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:07.932434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:51785 (local address 127.24.136.62:38915)
0809 22:47:07.933047 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:07.933053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:07.933336 (+ 283us) server_negotiation.cc:407] Connection header received
0809 22:47:07.933496 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:07.933500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:07.933575 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:07.933672 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:07.934673 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.935587 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.937184 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.937688 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.937789 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:07.943095 (+ 5306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:51785: BlockingRecv error: recv got EOF from 127.24.136.1:51785 (error 108)
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":333,"threads_started":1}
Aug 09 22:47:07 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:07 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:08.001246 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:07.991444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:47281)
0809 22:47:07.992053 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:07.992075 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:07.992227 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:07.992597 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:07.992602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:07.993142 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:07.993379 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:07.993391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.994642 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.994648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:07.995870 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:07.995885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.996631 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.996637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:07.996789 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:07.997405 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:07.997424 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:08.000896 (+ 3472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":303,"threads_started":1}
W20260809 22:47:08.001431 8836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:07.991678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:47281 (local address 127.24.136.62:38915)
0809 22:47:07.991926 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:07.991933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:07.992167 (+ 234us) server_negotiation.cc:407] Connection header received
0809 22:47:07.992355 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:07.992359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:07.992427 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:07.992516 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:07.993550 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.994499 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.996021 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:07.996443 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:07.996556 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:08.001180 (+ 4624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:47281: BlockingRecv error: recv got EOF from 127.24.136.3:47281 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 09 22:47:08 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:08 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:08.391808 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:08.379072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:39605)
0809 22:47:08.379693 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:08.379711 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:08.379860 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:08.380225 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:08.380230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:08.380668 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:08.380864 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:08.380874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:08.381934 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:08.381938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:08.382866 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:08.382879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:08.383633 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:08.383638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:08.383816 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:08.387963 (+ 4147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:08.387985 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:08.391548 (+ 3563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":378,"threads_started":1}
W20260809 22:47:08.392005 8836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:08.379243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:39605 (local address 127.24.136.62:38915)
0809 22:47:08.379520 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:08.379527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:08.379809 (+ 282us) server_negotiation.cc:407] Connection header received
0809 22:47:08.379985 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:08.379989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:08.380071 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:08.380157 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:08.381023 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:08.381792 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:08.383013 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:08.383426 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:08.383569 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:08.391753 (+ 8184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:39605: BlockingRecv error: recv got EOF from 127.24.136.2:39605 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 09 22:47:08 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:08 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:08.953899 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:08.944366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:34635)
0809 22:47:08.944977 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:08.944994 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:08.945125 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:08.945486 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:08.945490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:08.945944 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:08.946137 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:08.946147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:08.947178 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:08.947182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:08.948211 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:08.948224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:08.949003 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:08.949009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:08.949164 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:08.949845 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:08.949872 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:08.953632 (+ 3760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":349,"threads_started":1}
W20260809 22:47:08.954027 8844 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:08.944458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:34635 (local address 127.24.136.62:38915)
0809 22:47:08.944988 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:08.944993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:08.945110 (+ 117us) server_negotiation.cc:407] Connection header received
0809 22:47:08.945228 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:08.945231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:08.945307 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:08.945400 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:08.946267 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:08.947035 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:08.948370 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:08.948807 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:08.948907 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:08.953827 (+ 4920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:34635: BlockingRecv error: recv got EOF from 127.24.136.1:34635 (error 108)
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":278,"threads_started":1}
Aug 09 22:47:09 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:09 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:09.011353 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:09.002272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:34215)
0809 22:47:09.002743 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:09.002764 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:09.002885 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:09.003159 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:09.003162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:09.003683 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:09.003895 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:09.003905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.005076 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.005081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:09.006089 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:09.006109 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.006812 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.006817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:09.006973 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:09.007653 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:09.007672 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:09.011108 (+ 3436us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":242,"threads_started":1}
W20260809 22:47:09.011596 8844 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:09.002327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:34215 (local address 127.24.136.62:38915)
0809 22:47:09.002533 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:09.002541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:09.002843 (+ 302us) server_negotiation.cc:407] Connection header received
0809 22:47:09.002978 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:09.002982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:09.003048 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:09.003126 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:09.004073 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.004875 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.006233 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.006653 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.006756 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:09.011345 (+ 4589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:34215: BlockingRecv error: recv got EOF from 127.24.136.3:34215 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 22:47:09 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:09 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:09.402858 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:09.392998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:42159)
0809 22:47:09.393599 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:09.393617 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:09.393774 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:09.394176 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:09.394181 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:09.394683 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:09.394916 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:09.394931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.396205 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.396211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:09.397298 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:09.397313 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.398130 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.398136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:09.398287 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:09.398908 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:09.398927 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:09.402527 (+ 3600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":311,"threads_started":1}
W20260809 22:47:09.403061 8844 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:09.393066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:42159 (local address 127.24.136.62:38915)
0809 22:47:09.393302 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:09.393309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:09.393721 (+ 412us) server_negotiation.cc:407] Connection header received
0809 22:47:09.393919 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:09.393924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:09.393994 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:09.394084 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:09.395104 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.396070 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.397465 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.397930 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.398053 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:09.402780 (+ 4727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:42159: BlockingRecv error: recv got EOF from 127.24.136.2:42159 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 22:47:09 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:09 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:09.964262 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:09.954955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:58419)
0809 22:47:09.955517 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:09.955535 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:09.955663 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:09.956049 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:09.956052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:09.956463 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:09.956628 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:09.956661 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.957787 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.957792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:09.958742 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:09.958756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.959570 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.959575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:09.959731 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:09.960377 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:09.960396 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:09.964012 (+ 3616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":281,"threads_started":1}
W20260809 22:47:09.964560 8849 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:09.955085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:58419 (local address 127.24.136.62:38915)
0809 22:47:09.955606 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:09.955611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:09.955629 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:47:09.955826 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:09.955830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:09.955897 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:09.955989 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:09.956845 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.957612 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.958901 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:09.959367 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:09.959493 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:09.964288 (+ 4795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:58419: BlockingRecv error: recv got EOF from 127.24.136.1:58419 (error 108)
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":255,"threads_started":1}
Aug 09 22:47:10 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:10 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:10.022131 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:10.012748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:41551)
0809 22:47:10.013255 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:10.013276 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:10.013412 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:10.013799 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:10.013803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:10.014209 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:10.014389 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:10.014399 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.015546 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.015551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:10.016591 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:10.016608 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.017479 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.017485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:10.017712 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:10.018355 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:10.018374 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:10.021882 (+ 3508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":266,"threads_started":1}
W20260809 22:47:10.022358 8849 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:10.012847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:41551 (local address 127.24.136.62:38915)
0809 22:47:10.013081 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:10.013104 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:10.013387 (+ 283us) server_negotiation.cc:407] Connection header received
0809 22:47:10.013520 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:10.013524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:10.013597 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:10.013688 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:10.014568 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.015341 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.016754 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.017324 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.017463 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:10.022124 (+ 4661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:41551: BlockingRecv error: recv got EOF from 127.24.136.3:41551 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 22:47:10 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:10 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:10.413719 8852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:10.404152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:54381)
0809 22:47:10.404688 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:10.404705 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:10.404838 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:10.405260 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:10.405263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:10.405790 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:10.405975 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:10.405986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.407324 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.407329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:10.408565 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:10.408580 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.409263 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.409267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:10.409387 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:10.409992 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:10.410015 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:10.413456 (+ 3441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":229,"threads_started":1}
W20260809 22:47:10.414000 8849 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:10.404382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:54381 (local address 127.24.136.62:38915)
0809 22:47:10.404659 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:10.404664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:10.404778 (+ 114us) server_negotiation.cc:407] Connection header received
0809 22:47:10.405042 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:10.405048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:10.405125 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:10.405223 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:10.406131 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.407159 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.408722 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.409145 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.409263 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:10.413720 (+ 4457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:54381: BlockingRecv error: recv got EOF from 127.24.136.2:54381 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 09 22:47:10 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:10 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.1@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:10.975677 8853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:10.965684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.1:59157)
0809 22:47:10.966252 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:10.966271 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:10.966387 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:10.966756 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:10.966759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:10.967144 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:10.967350 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:10.967360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.968569 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.968573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:10.969794 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:10.969812 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.970619 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.970625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:10.970823 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:10.971440 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:10.971501 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:10.975321 (+ 3820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":291,"threads_started":1}
W20260809 22:47:10.975919 8854 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:10.965873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:59157 (local address 127.24.136.62:38915)
0809 22:47:10.966425 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:10.966430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:10.966451 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:47:10.966513 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:10.966517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:10.966611 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:10.966722 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:10.967572 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.968379 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.969944 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:10.970441 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:10.970543 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:10.975672 (+ 5129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.1:59157: BlockingRecv error: recv got EOF from 127.24.136.1:59157 (error 108)
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":324,"threads_started":1}
Aug 09 22:47:11 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:11 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.3@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:11.032518 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:11.023173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.3:43199)
0809 22:47:11.023744 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:11.023765 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:11.023906 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:11.024243 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:11.024248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:11.024698 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:11.024863 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:11.024874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.025916 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.025920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:11.026860 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:11.026871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.027623 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.027629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:11.027818 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:11.028444 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:11.028463 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:11.032171 (+ 3708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":294,"threads_started":1}
W20260809 22:47:11.032682 8854 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:11.023270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:43199 (local address 127.24.136.62:38915)
0809 22:47:11.023645 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:11.023651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:11.023850 (+ 199us) server_negotiation.cc:407] Connection header received
0809 22:47:11.024029 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:11.024033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:11.024099 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:11.024186 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:11.025021 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.025762 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.027028 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.027444 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.027600 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:11.032437 (+ 4837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.3:43199: BlockingRecv error: recv got EOF from 127.24.136.3:43199 (error 108)
Metrics: {"server-negotiator.queue_time_us":218}
Aug 09 22:47:11 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:47:11 dist-test-slave-pww0 krb5kdc[7044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.136.2@KRBTEST.COM for oryx/127.24.136.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:47:11.423923 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:11.415029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38915 (local address 127.24.136.2:52861)
0809 22:47:11.415537 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:11.415553 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:11.415681 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:11.415973 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:11.415977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:11.416342 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:11.416525 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:11.416533 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.417513 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.417517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:11.418458 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:11.418470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.419222 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.419227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:11.419382 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:11.420091 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:11.420113 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:11.423640 (+ 3527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.136.62:38915: Server oryx/127.24.136.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":230,"threads_started":1}
W20260809 22:47:11.424153 8854 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:11.415104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:52861 (local address 127.24.136.62:38915)
0809 22:47:11.415307 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:11.415313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:11.415622 (+ 309us) server_negotiation.cc:407] Connection header received
0809 22:47:11.415785 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:11.415789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:11.415869 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:11.415944 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:11.416638 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.417375 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.418621 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.419053 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.419160 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:11.423918 (+ 4758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.136.2:52861: BlockingRecv error: recv got EOF from 127.24.136.2:52861 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:47:11.636622 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 8074
I20260809 22:47:11.649950 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:11.637946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.24.136.1:45569)
0809 22:47:11.638609 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:11.638616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:11.638654 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:47:11.638720 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:11.638727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:11.639013 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:11.639195 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:11.640790 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.641906 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.643202 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.643902 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.644047 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:11.646609 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:11.646618 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:11.646639 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:11.646706 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:11.649362 (+ 2656us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.136.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 22:47:11.649727 (+ 365us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39562: Request ticket server kudu/127.24.136.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":309,"threads_started":1}
W20260809 22:47:11.650066 8859 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39562: Request ticket server kudu/127.24.136.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 22:47:11.650357 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 8074: Not authorized: Client connection negotiation failed: client connection to 127.24.136.1:45569: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.136.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 22:47:11.650554 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 8074
I20260809 22:47:11.677634 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 8321
I20260809 22:47:11.690972 8860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:11.678958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.24.136.2:40511)
0809 22:47:11.679677 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:11.679685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:11.679718 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:47:11.679791 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:11.679797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:11.680055 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:11.680264 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:11.681658 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.682737 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.684091 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.684699 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.684824 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:11.687383 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:11.687392 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:11.687409 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:11.687481 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:11.690212 (+ 2731us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.136.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 22:47:11.690620 (+ 408us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39816: Request ticket server kudu/127.24.136.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":277,"threads_started":1}
W20260809 22:47:11.691084 8860 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39816: Request ticket server kudu/127.24.136.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 22:47:11.691197 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 8321: Not authorized: Client connection negotiation failed: client connection to 127.24.136.2:40511: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.136.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 22:47:11.691358 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 8321
I20260809 22:47:11.714999 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 8546
I20260809 22:47:11.728941 8861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:11.716318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.24.136.3:33217)
0809 22:47:11.716990 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:11.716998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:11.717027 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:47:11.717102 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:11.717110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:11.717358 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:11.717541 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:11.719414 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.720697 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.721958 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:11.722536 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:11.722651 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:11.725194 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:11.725206 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:11.725225 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:11.725281 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:11.728302 (+ 3021us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.136.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 22:47:11.728669 (+ 367us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42626: Request ticket server kudu/127.24.136.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":362,"threads_started":1}
W20260809 22:47:11.729075 8861 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42626: Request ticket server kudu/127.24.136.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 22:47:11.729139 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 8546: Not authorized: Client connection negotiation failed: client connection to 127.24.136.3:33217: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.136.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 22:47:11.729348 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 8546
I20260809 22:47:11.760973 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 7956
I20260809 22:47:11.762441 8064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:11.879148 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 7956
2026-08-09T22:47:11Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21928 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T22:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:47:11Z Disabled control of system clock
I20260809 22:47:11.979776 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38903
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 22:47:12.338917 8873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:12.339262 8873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:12.339378 8873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:12.347000 8873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:47:12.347139 8873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:12.347183 8873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:12.347266 8873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:47:12.347326 8873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:47:12.357012 8873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:12.358430 8873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:12.360335 8873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:12.372588 8886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:12.372548 8882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:12.372509 8883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:12.373518 8873 server_base.cc:1034] running on GCE node
I20260809 22:47:12.374352 8873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:12.375919 8873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:12.377152 8873 hybrid_clock.cc:648] HybridClock initialized: now 1786315632377093 us; error 36 us; skew 500 ppm
I20260809 22:47:12.377522 8873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:47:12.379942 8873 webserver.cc:460] Webserver started at http://127.24.136.62:35317/ using document root <none> and password file <none>
I20260809 22:47:12.380677 8873 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:12.380772 8873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:12.381104 8873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:12.384172 8873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "c14ba2ed214046089643f02a70829d19"
format_stamp: "Formatted at 2026-08-09 22:47:12 on dist-test-slave-pww0"
server_key: "b66450c28926156374045acbf351e0d1"
server_key_iv: "4a260cfb0f8bb97916b11f7438e2e256"
server_key_version: "encryptionkey@0"
I20260809 22:47:12.385021 8873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "c14ba2ed214046089643f02a70829d19"
format_stamp: "Formatted at 2026-08-09 22:47:12 on dist-test-slave-pww0"
server_key: "b66450c28926156374045acbf351e0d1"
server_key_iv: "4a260cfb0f8bb97916b11f7438e2e256"
server_key_version: "encryptionkey@0"
I20260809 22:47:12.390542 8873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:47:12.394264 8898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:12.395949 8873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:47:12.396160 8873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "c14ba2ed214046089643f02a70829d19"
format_stamp: "Formatted at 2026-08-09 22:47:12 on dist-test-slave-pww0"
server_key: "b66450c28926156374045acbf351e0d1"
server_key_iv: "4a260cfb0f8bb97916b11f7438e2e256"
server_key_version: "encryptionkey@0"
I20260809 22:47:12.396394 8873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:12.410498 8873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:12.411412 8873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:12.411751 8873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:12.430033 8873 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38903
I20260809 22:47:12.430051 8982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38903 every 8 connection(s)
I20260809 22:47:12.431507 8873 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:47:12.435758 8986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:12.439965 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 8873
I20260809 22:47:12.440168 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:47:12.440981 25120 external_mini_cluster.cc:1428] Setting key 9c4e7ae8a30c3f495e2e70e1d97bcafb
I20260809 22:47:12.451565 8986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19: Bootstrap starting.
I20260809 22:47:12.456159 8986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:12.457809 8986 log.cc:824] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:12.457983 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:12.443354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.24.136.62:38903)
0809 22:47:12.444592 (+ 1238us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:12.444610 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:12.444669 (+ 59us) server_negotiation.cc:407] Connection header received
0809 22:47:12.446526 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:12.446561 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:12.447283 (+ 722us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:12.447814 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:12.448448 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:12.450461 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:12.451954 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:12.453099 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:12.453328 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:12.454683 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:12.454696 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:12.454716 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:12.455562 (+ 846us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:12.455707 (+ 145us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:12.455742 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:12.455890 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:12.456114 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 22:47:12.456505 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":374,"threads_started":1}
I20260809 22:47:12.461382 8986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19: No bootstrap required, opened a new log
I20260809 22:47:12.469308 8986 raft_consensus.cc:348] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14ba2ed214046089643f02a70829d19" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38903 } }
I20260809 22:47:12.469718 8986 raft_consensus.cc:374] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:12.469798 8986 raft_consensus.cc:729] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14ba2ed214046089643f02a70829d19, State: Initialized, Role: FOLLOWER
I20260809 22:47:12.470414 8986 consensus_queue.cc:260] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14ba2ed214046089643f02a70829d19" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38903 } }
I20260809 22:47:12.470726 8986 raft_consensus.cc:388] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:47:12.470871 8986 raft_consensus.cc:482] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:47:12.471040 8986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:12.473634 8986 raft_consensus.cc:504] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14ba2ed214046089643f02a70829d19" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38903 } }
I20260809 22:47:12.474226 8986 leader_election.cc:302] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c14ba2ed214046089643f02a70829d19; no voters:
I20260809 22:47:12.474864 8986 leader_election.cc:288] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:47:12.475021 8991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:47:12.475776 8991 raft_consensus.cc:686] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [term 1 LEADER]: Becoming Leader. State: Replica: c14ba2ed214046089643f02a70829d19, State: Running, Role: LEADER
I20260809 22:47:12.476363 8991 consensus_queue.cc:237] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14ba2ed214046089643f02a70829d19" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38903 } }
I20260809 22:47:12.476999 8986 sys_catalog.cc:564] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:47:12.480137 8993 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c14ba2ed214046089643f02a70829d19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14ba2ed214046089643f02a70829d19" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38903 } } }
I20260809 22:47:12.480584 8993 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [sys.catalog]: This master's current role is: LEADER
I20260809 22:47:12.481047 8995 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c14ba2ed214046089643f02a70829d19. Latest consensus state: current_term: 1 leader_uuid: "c14ba2ed214046089643f02a70829d19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c14ba2ed214046089643f02a70829d19" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38903 } } }
I20260809 22:47:12.481393 8995 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19 [sys.catalog]: This master's current role is: LEADER
I20260809 22:47:12.482995 9002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:47:12.488994 9002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:47:12.499018 9002 catalog_manager.cc:1347] Generated new cluster ID: 30b18991fde24544a9cbfa2f61f7af4f
I20260809 22:47:12.499155 9002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:47:12.517930 9002 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:47:12.519392 9002 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:47:12.527760 9002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c14ba2ed214046089643f02a70829d19: Generated new TSK 0
I20260809 22:47:12.528752 9002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:47:12.534212 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38903
--builtin_ntp_servers=127.24.136.20:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 22:47:12.890739 9021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:12.891057 9021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:12.891175 9021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:12.899044 9021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:12.899168 9021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:12.899302 9021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:47:12.909377 9021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:12.910861 9021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:12.912915 9021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:12.926563 9027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:12.926616 9030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:12.926630 9035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:12.927491 9021 server_base.cc:1034] running on GCE node
I20260809 22:47:12.928174 9021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:12.929226 9021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:12.930454 9021 hybrid_clock.cc:648] HybridClock initialized: now 1786315632930394 us; error 45 us; skew 500 ppm
I20260809 22:47:12.930814 9021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:47:12.933238 9021 webserver.cc:460] Webserver started at http://127.24.136.1:45995/ using document root <none> and password file <none>
I20260809 22:47:12.934003 9021 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:12.934122 9021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:12.934484 9021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:12.937701 9021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "aa61ff30dcb94a8b83b08fa1566d84bb"
format_stamp: "Formatted at 2026-08-09 22:47:12 on dist-test-slave-pww0"
server_key: "cf9afde89792c0effd27b8d95f0b71c4"
server_key_iv: "5fc95ef56f041defcee8e4a482a52618"
server_key_version: "encryptionkey@0"
I20260809 22:47:12.938607 9021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "aa61ff30dcb94a8b83b08fa1566d84bb"
format_stamp: "Formatted at 2026-08-09 22:47:12 on dist-test-slave-pww0"
server_key: "cf9afde89792c0effd27b8d95f0b71c4"
server_key_iv: "5fc95ef56f041defcee8e4a482a52618"
server_key_version: "encryptionkey@0"
I20260809 22:47:12.944576 9021 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 22:47:12.948552 9044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:12.949918 9021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:47:12.950096 9021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "aa61ff30dcb94a8b83b08fa1566d84bb"
format_stamp: "Formatted at 2026-08-09 22:47:12 on dist-test-slave-pww0"
server_key: "cf9afde89792c0effd27b8d95f0b71c4"
server_key_iv: "5fc95ef56f041defcee8e4a482a52618"
server_key_version: "encryptionkey@0"
I20260809 22:47:12.950327 9021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:12.973093 9021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:12.974099 9021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:12.974428 9021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:12.975528 9021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:12.977236 9021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:12.977344 9021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:12.977474 9021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:12.977558 9021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:13.010231 9021 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:34305
I20260809 22:47:13.010326 9241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:34305 every 8 connection(s)
I20260809 22:47:13.011755 9021 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:47:13.015724 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 9021
I20260809 22:47:13.015969 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:47:13.016873 25120 external_mini_cluster.cc:1428] Setting key e5b0d7c2bdb8eac5d70d92f375215bee
I20260809 22:47:13.019876 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38903
--builtin_ntp_servers=127.24.136.20:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 22:47:13.025836 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:13.014604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:51109 (local address 127.24.136.62:38903)
0809 22:47:13.015136 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:13.015141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:13.016687 (+ 1546us) server_negotiation.cc:407] Connection header received
0809 22:47:13.018645 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:13.018650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:13.018737 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:13.018833 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:13.020179 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:13.021191 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:13.022844 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:13.023387 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:13.023554 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:13.024934 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:13.024941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:13.024945 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:13.024969 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:13.025025 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:13.025040 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:13.025128 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:13.025405 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 22:47:13.025505 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":263,"threads_started":1}
I20260809 22:47:13.027036 9251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:13.015359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38903 (local address 127.24.136.1:51109)
0809 22:47:13.016522 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:13.016560 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:13.018366 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:13.019096 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:13.019106 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:13.019206 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:13.019897 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:13.019915 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:13.021370 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:13.021375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:13.022688 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:13.022702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:13.023642 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:13.023665 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:13.023899 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:13.024589 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:47:13.024591 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:13.024635 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:47:13.024739 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:13.025170 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:13.025181 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:13.025188 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 22:47:13.025380 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 22:47:13.025749 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":392,"threads_started":1}
I20260809 22:47:13.029078 9246 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38903
I20260809 22:47:13.029448 9246 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:13.030354 9246 heartbeater.cc:511] Master 127.24.136.62:38903 requested a full tablet report, sending...
I20260809 22:47:13.032588 8927 ts_manager.cc:194] Registered new tserver with Master: aa61ff30dcb94a8b83b08fa1566d84bb (127.24.136.1:34305)
I20260809 22:47:13.034077 8927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.1:51109
W20260809 22:47:13.373356 9254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:13.373696 9254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:13.373826 9254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:13.381474 9254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:13.381609 9254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:13.381762 9254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:47:13.391223 9254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:13.392656 9254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:13.394517 9254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:13.408044 9261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:13.408130 9262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:13.408083 9269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:13.408941 9254 server_base.cc:1034] running on GCE node
I20260809 22:47:13.409583 9254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:13.410661 9254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:13.411888 9254 hybrid_clock.cc:648] HybridClock initialized: now 1786315633411829 us; error 52 us; skew 500 ppm
I20260809 22:47:13.412223 9254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:47:13.418778 9254 webserver.cc:460] Webserver started at http://127.24.136.2:39121/ using document root <none> and password file <none>
I20260809 22:47:13.419624 9254 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:13.419754 9254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:13.420097 9254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:13.423094 9254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "411f893040874ddd939afaa9e975eb1f"
format_stamp: "Formatted at 2026-08-09 22:47:13 on dist-test-slave-pww0"
server_key: "97b26cfe76a661037ce489f24db4c6ae"
server_key_iv: "ed648e13b81699fcb1dae42ece7fa737"
server_key_version: "encryptionkey@0"
I20260809 22:47:13.423974 9254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "411f893040874ddd939afaa9e975eb1f"
format_stamp: "Formatted at 2026-08-09 22:47:13 on dist-test-slave-pww0"
server_key: "97b26cfe76a661037ce489f24db4c6ae"
server_key_iv: "ed648e13b81699fcb1dae42ece7fa737"
server_key_version: "encryptionkey@0"
I20260809 22:47:13.429546 9254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:47:13.433558 9281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:13.435101 9254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:47:13.435312 9254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "411f893040874ddd939afaa9e975eb1f"
format_stamp: "Formatted at 2026-08-09 22:47:13 on dist-test-slave-pww0"
server_key: "97b26cfe76a661037ce489f24db4c6ae"
server_key_iv: "ed648e13b81699fcb1dae42ece7fa737"
server_key_version: "encryptionkey@0"
I20260809 22:47:13.435590 9254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:13.492659 9254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:13.493525 9254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:13.493808 9254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:13.494807 9254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:13.496421 9254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:13.496512 9254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:13.496595 9254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:13.496656 9254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:13.529107 9254 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:44573
I20260809 22:47:13.529191 9478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:44573 every 8 connection(s)
I20260809 22:47:13.530515 9254 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:47:13.531075 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 9254
I20260809 22:47:13.531275 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:47:13.531921 25120 external_mini_cluster.cc:1428] Setting key bd9846d45c8c4b2956cea3d8679eec84
I20260809 22:47:13.535312 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38903
--builtin_ntp_servers=127.24.136.20:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 22:47:13.544993 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:13.533602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:60985 (local address 127.24.136.62:38903)
0809 22:47:13.534049 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:13.534055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:13.535951 (+ 1896us) server_negotiation.cc:407] Connection header received
0809 22:47:13.537905 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:13.537911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:13.538009 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:13.538132 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:13.539441 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:13.540434 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:13.542065 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:13.542606 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:13.542743 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:13.544170 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:13.544176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:13.544180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:13.544206 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:13.544264 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:13.544279 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:13.544360 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:13.544551 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 22:47:13.544665 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":223,"threads_started":1}
I20260809 22:47:13.545979 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:13.534491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38903 (local address 127.24.136.2:60985)
0809 22:47:13.535753 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:13.535808 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:13.537587 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:13.538421 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:13.538430 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:13.538536 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:13.539257 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:13.539279 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:13.540599 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:13.540606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:13.541913 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:13.541926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:13.542839 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:13.542844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:13.543055 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:13.543822 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:47:13.543824 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:13.543870 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:47:13.543965 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:13.544368 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:13.544375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:13.544380 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:47:13.544523 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 22:47:13.544875 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":349,"threads_started":1}
I20260809 22:47:13.547503 9480 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38903
I20260809 22:47:13.547863 9480 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:13.548782 9480 heartbeater.cc:511] Master 127.24.136.62:38903 requested a full tablet report, sending...
I20260809 22:47:13.550542 8927 ts_manager.cc:194] Registered new tserver with Master: 411f893040874ddd939afaa9e975eb1f (127.24.136.2:44573)
I20260809 22:47:13.551608 8927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.2:60985
W20260809 22:47:13.884636 9489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:13.884938 9489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:13.885025 9489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:13.892464 9489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:13.892617 9489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:13.892740 9489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:47:13.902235 9489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:13.903553 9489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:13.905395 9489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:13.918918 9500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:13.918994 9498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:13.918958 9502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:13.919927 9489 server_base.cc:1034] running on GCE node
I20260809 22:47:13.920603 9489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:13.921639 9489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:13.922857 9489 hybrid_clock.cc:648] HybridClock initialized: now 1786315633922807 us; error 44 us; skew 500 ppm
I20260809 22:47:13.923209 9489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:47:13.925537 9489 webserver.cc:460] Webserver started at http://127.24.136.3:39913/ using document root <none> and password file <none>
I20260809 22:47:13.926298 9489 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:13.926393 9489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:13.926721 9489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:13.929654 9489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "eb0b0fb75ecd459d85f4bae2fda43151"
format_stamp: "Formatted at 2026-08-09 22:47:13 on dist-test-slave-pww0"
server_key: "a468338423aa367883516a7f03337b41"
server_key_iv: "9383a3a8c1eb6fb9fc4eb85ddd91a068"
server_key_version: "encryptionkey@0"
I20260809 22:47:13.930481 9489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "eb0b0fb75ecd459d85f4bae2fda43151"
format_stamp: "Formatted at 2026-08-09 22:47:13 on dist-test-slave-pww0"
server_key: "a468338423aa367883516a7f03337b41"
server_key_iv: "9383a3a8c1eb6fb9fc4eb85ddd91a068"
server_key_version: "encryptionkey@0"
I20260809 22:47:13.936079 9489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:47:13.940083 9512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:13.941535 9489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:47:13.941736 9489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "eb0b0fb75ecd459d85f4bae2fda43151"
format_stamp: "Formatted at 2026-08-09 22:47:13 on dist-test-slave-pww0"
server_key: "a468338423aa367883516a7f03337b41"
server_key_iv: "9383a3a8c1eb6fb9fc4eb85ddd91a068"
server_key_version: "encryptionkey@0"
I20260809 22:47:13.941975 9489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:13.956841 9489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:13.957769 9489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:13.958079 9489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:13.959077 9489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:13.960692 9489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:13.960795 9489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:13.960911 9489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:13.960983 9489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:13.992730 9489 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:39857
I20260809 22:47:13.992785 9699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:39857 every 8 connection(s)
I20260809 22:47:13.994082 9489 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:47:13.998517 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 9489
I20260809 22:47:13.998701 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:47:13.999285 25120 external_mini_cluster.cc:1428] Setting key 8e4219ae09801c52a97b40552919516b
I20260809 22:47:14.007637 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:13.996722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:38217 (local address 127.24.136.62:38903)
0809 22:47:13.996934 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:13.996940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:13.998442 (+ 1502us) server_negotiation.cc:407] Connection header received
0809 22:47:14.000723 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:14.000728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:14.000797 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:14.000876 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:14.002116 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.003090 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.004863 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.005284 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.005395 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:14.006866 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:14.006874 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:14.006879 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:14.006906 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:14.006949 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:14.006966 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:14.007039 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:14.007270 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 22:47:14.007361 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:47:14.008771 9702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:13.997245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38903 (local address 127.24.136.3:38217)
0809 22:47:13.998251 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:13.998310 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:14.000461 (+ 2151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:14.001165 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:14.001173 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:14.001272 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:14.001943 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:14.001961 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.003265 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.003270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:14.004697 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:14.004713 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.005506 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.005511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:14.005763 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:14.006485 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:47:14.006488 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:14.006533 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:47:14.006626 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:14.007058 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:14.007068 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:14.007075 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 22:47:14.007255 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 22:47:14.007669 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":289,"threads_started":1}
I20260809 22:47:14.010331 9700 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38903
I20260809 22:47:14.010654 9700 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:14.011546 9700 heartbeater.cc:511] Master 127.24.136.62:38903 requested a full tablet report, sending...
I20260809 22:47:14.013070 8927 ts_manager.cc:194] Registered new tserver with Master: eb0b0fb75ecd459d85f4bae2fda43151 (127.24.136.3:39857)
I20260809 22:47:14.014217 8927 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.3:38217
I20260809 22:47:14.015221 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 22:47:14.022243 9485 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:14.020571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.24.136.62:38903)
0809 22:47:14.020826 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:14.020834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:14.021217 (+ 383us) server_negotiation.cc:407] Connection header received
0809 22:47:14.021367 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:14.021371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:14.021443 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:14.021532 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:14.021962 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54022: BlockingRecv error: recv got EOF from 127.0.0.1:54022 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 22:47:14.022630 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 9021
I20260809 22:47:14.031164 9714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:14.023924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46200 (local address 127.24.136.1:34305)
0809 22:47:14.024420 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:14.024427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:14.024456 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:47:14.024521 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:14.024527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:14.024744 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:14.024879 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:14.025263 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.026321 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.027847 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.028494 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.028628 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:14.029766 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:14.029775 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:14.029789 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:14.030392 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:14.030447 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:14.030470 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:14.030561 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:14.030736 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 22:47:14.030865 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":230,"threads_started":1}
I20260809 22:47:14.032568 9235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:14.118228 9246 heartbeater.cc:503] Master 127.24.136.62:38903 was elected leader, sending a full tablet report...
I20260809 22:47:14.119539 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 9021
W20260809 22:47:14.146981 8901 connection.cc:535] server connection from 127.24.136.1:51109 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 22:47:14.147526 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 9254
I20260809 22:47:14.158718 9717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:14.148519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.24.136.2:44573)
0809 22:47:14.150121 (+ 1602us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:14.150129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:14.150159 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:47:14.150228 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:14.150235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:14.150537 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:14.150729 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:14.151906 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.153243 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.155140 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.155886 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.156017 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:14.157137 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:14.157147 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:14.157162 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:14.157805 (+ 643us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:14.157859 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:14.157880 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:14.157988 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:14.158168 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 22:47:14.158320 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1442,"thread_start_us":1276,"threads_started":1}
I20260809 22:47:14.160737 9472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:14.244879 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 9254
I20260809 22:47:14.274740 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 9489
I20260809 22:47:14.283610 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:14.275893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.24.136.3:39857)
0809 22:47:14.276623 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:14.276633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:14.276668 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:47:14.276737 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:14.276744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:14.276962 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:14.277098 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:14.277635 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.278824 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.280219 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.280823 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.280924 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:14.281927 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:14.281936 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:14.281952 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:14.282588 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:14.282643 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:14.282670 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:14.282773 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:14.283023 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 22:47:14.283178 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":399,"threads_started":1}
I20260809 22:47:14.284981 9690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:14.367908 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 9489
I20260809 22:47:14.391166 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 8873
I20260809 22:47:14.392463 8976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:14.497491 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 8873
2026-08-09T22:47:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2607 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T22:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:47:14Z Disabled control of system clock
I20260809 22:47:14.587697 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:34237
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:43229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:34237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 22:47:14.942982 9734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:14.943313 9734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:14.943405 9734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:14.950989 9734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:47:14.951139 9734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:14.951211 9734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:14.951292 9734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:47:14.951352 9734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:47:14.961063 9734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:43229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:34237
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:34237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:14.962468 9734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:14.964408 9734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:14.977571 9748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:14.977479 9742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:14.977535 9744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:14.978516 9734 server_base.cc:1034] running on GCE node
I20260809 22:47:14.979421 9734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:14.981158 9734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:14.982406 9734 hybrid_clock.cc:648] HybridClock initialized: now 1786315634982354 us; error 46 us; skew 500 ppm
I20260809 22:47:14.982842 9734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:47:14.985579 9734 webserver.cc:460] Webserver started at http://127.24.136.62:43037/ using document root <none> and password file <none>
I20260809 22:47:14.986472 9734 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:14.986609 9734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:14.986975 9734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:14.992573 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "c4b3f02c366e42e0a7b28116ffd6b07e"
format_stamp: "Formatted at 2026-08-09 22:47:14 on dist-test-slave-pww0"
server_key: "de0dfe33e6d7a864529528ece15d7c4c"
server_key_iv: "d94add8c9de2ae469cce9689beb3750d"
server_key_version: "encryptionkey@0"
I20260809 22:47:14.993510 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "c4b3f02c366e42e0a7b28116ffd6b07e"
format_stamp: "Formatted at 2026-08-09 22:47:14 on dist-test-slave-pww0"
server_key: "de0dfe33e6d7a864529528ece15d7c4c"
server_key_iv: "d94add8c9de2ae469cce9689beb3750d"
server_key_version: "encryptionkey@0"
I20260809 22:47:14.999496 9734 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260809 22:47:15.003420 9758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:15.005142 9734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:47:15.005366 9734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "c4b3f02c366e42e0a7b28116ffd6b07e"
format_stamp: "Formatted at 2026-08-09 22:47:14 on dist-test-slave-pww0"
server_key: "de0dfe33e6d7a864529528ece15d7c4c"
server_key_iv: "d94add8c9de2ae469cce9689beb3750d"
server_key_version: "encryptionkey@0"
I20260809 22:47:15.005614 9734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:15.012403 9734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:15.013350 9734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:15.013679 9734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:15.032848 9734 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:34237
I20260809 22:47:15.032894 9837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:34237 every 8 connection(s)
I20260809 22:47:15.034304 9734 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:47:15.036867 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 9734
I20260809 22:47:15.037108 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:47:15.037930 25120 external_mini_cluster.cc:1428] Setting key f427d419ccfd824e78bf02c6cb775666
I20260809 22:47:15.039237 9838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:15.047698 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:15.040292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.24.136.62:34237)
0809 22:47:15.041634 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:15.041645 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:15.041695 (+ 50us) server_negotiation.cc:407] Connection header received
0809 22:47:15.043517 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:15.043547 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:15.044081 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:15.044529 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:15.045772 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:15.045780 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:15.045794 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:15.045811 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:15.045905 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:15.045927 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:15.046196 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:15.046355 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 22:47:15.046666 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":448,"threads_started":1}
I20260809 22:47:15.053334 9838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e: Bootstrap starting.
I20260809 22:47:15.056871 9838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:15.058030 9838 log.cc:824] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:15.061497 9838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e: No bootstrap required, opened a new log
I20260809 22:47:15.069463 9838 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b3f02c366e42e0a7b28116ffd6b07e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34237 } }
I20260809 22:47:15.069875 9838 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:15.069983 9838 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b3f02c366e42e0a7b28116ffd6b07e, State: Initialized, Role: FOLLOWER
I20260809 22:47:15.070608 9838 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b3f02c366e42e0a7b28116ffd6b07e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34237 } }
I20260809 22:47:15.070834 9838 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:47:15.070968 9838 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:47:15.071135 9838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:15.073415 9838 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b3f02c366e42e0a7b28116ffd6b07e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34237 } }
I20260809 22:47:15.073979 9838 leader_election.cc:302] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4b3f02c366e42e0a7b28116ffd6b07e; no voters:
I20260809 22:47:15.074612 9838 leader_election.cc:288] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:47:15.074781 9848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:47:15.075537 9848 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [term 1 LEADER]: Becoming Leader. State: Replica: c4b3f02c366e42e0a7b28116ffd6b07e, State: Running, Role: LEADER
I20260809 22:47:15.076176 9848 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b3f02c366e42e0a7b28116ffd6b07e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34237 } }
I20260809 22:47:15.076894 9838 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:47:15.079200 9851 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4b3f02c366e42e0a7b28116ffd6b07e. Latest consensus state: current_term: 1 leader_uuid: "c4b3f02c366e42e0a7b28116ffd6b07e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b3f02c366e42e0a7b28116ffd6b07e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34237 } } }
I20260809 22:47:15.079674 9851 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [sys.catalog]: This master's current role is: LEADER
I20260809 22:47:15.079860 9850 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4b3f02c366e42e0a7b28116ffd6b07e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4b3f02c366e42e0a7b28116ffd6b07e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 34237 } } }
I20260809 22:47:15.080399 9850 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e [sys.catalog]: This master's current role is: LEADER
I20260809 22:47:15.082063 9863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:47:15.087630 9863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:47:15.097700 9863 catalog_manager.cc:1347] Generated new cluster ID: 85020f5cff124e7a94dd106e5dbc3411
I20260809 22:47:15.097829 9863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:47:15.110742 9863 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:47:15.111255 9863 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:47:15.124400 9863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4b3f02c366e42e0a7b28116ffd6b07e: Generated new TSK 0
I20260809 22:47:15.125646 9863 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:47:15.133473 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34237
--builtin_ntp_servers=127.24.136.20:43229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 22:47:15.499833 9880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:15.500147 9880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:15.500268 9880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:15.507898 9880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:15.508025 9880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:15.508183 9880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:47:15.518021 9880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:43229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:15.519397 9880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:15.521263 9880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:15.534787 9892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:15.534754 9893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:15.534718 9896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:15.535874 9880 server_base.cc:1034] running on GCE node
I20260809 22:47:15.536595 9880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:15.537683 9880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:15.538909 9880 hybrid_clock.cc:648] HybridClock initialized: now 1786315635538856 us; error 53 us; skew 500 ppm
I20260809 22:47:15.539268 9880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:47:15.541970 9880 webserver.cc:460] Webserver started at http://127.24.136.1:37597/ using document root <none> and password file <none>
I20260809 22:47:15.542757 9880 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:15.542891 9880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:15.543238 9880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:15.546414 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "b624d53d8a0f413ca1a3dcf3ec0fa193"
format_stamp: "Formatted at 2026-08-09 22:47:15 on dist-test-slave-pww0"
server_key: "a231fafc678e429f57324eb9dc973370"
server_key_iv: "fecfc9a67eacf60008881f3e904e2f12"
server_key_version: "encryptionkey@0"
I20260809 22:47:15.547320 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "b624d53d8a0f413ca1a3dcf3ec0fa193"
format_stamp: "Formatted at 2026-08-09 22:47:15 on dist-test-slave-pww0"
server_key: "a231fafc678e429f57324eb9dc973370"
server_key_iv: "fecfc9a67eacf60008881f3e904e2f12"
server_key_version: "encryptionkey@0"
I20260809 22:47:15.553161 9880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:47:15.557085 9905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:15.558705 9880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:47:15.558902 9880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "b624d53d8a0f413ca1a3dcf3ec0fa193"
format_stamp: "Formatted at 2026-08-09 22:47:15 on dist-test-slave-pww0"
server_key: "a231fafc678e429f57324eb9dc973370"
server_key_iv: "fecfc9a67eacf60008881f3e904e2f12"
server_key_version: "encryptionkey@0"
I20260809 22:47:15.559146 9880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:15.565292 9880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:15.566188 9880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:15.566462 9880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:15.567608 9880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:15.569229 9880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:15.569324 9880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:15.569438 9880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:15.569511 9880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:15.602206 9880 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:43839
I20260809 22:47:15.602262 10092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:43839 every 8 connection(s)
I20260809 22:47:15.603688 9880 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:47:15.608530 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 9880
I20260809 22:47:15.608747 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:47:15.609448 25120 external_mini_cluster.cc:1428] Setting key 881bd0d64da468b57d186493f6bd195a
I20260809 22:47:15.612635 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34237
--builtin_ntp_servers=127.24.136.20:43229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 22:47:15.613606 10099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:15.606441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:36785 (local address 127.24.136.62:34237)
0809 22:47:15.607040 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:15.607047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:15.608613 (+ 1566us) server_negotiation.cc:407] Connection header received
0809 22:47:15.610815 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:15.610821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:15.610890 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:15.610995 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:15.612574 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:15.612677 (+ 103us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:15.612685 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:15.612720 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:15.612783 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:15.612807 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:15.612957 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:15.613212 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 22:47:15.613288 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":323,"threads_started":1}
I20260809 22:47:15.614660 10100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:15.607013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34237 (local address 127.24.136.1:36785)
0809 22:47:15.608408 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:15.608450 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:15.610469 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:15.611324 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:15.611338 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:15.611525 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:15.612225 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:47:15.612227 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:15.612278 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:47:15.612382 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:15.612937 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:15.612950 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:15.612959 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:47:15.613276 (+ 317us) client_negotiation.cc:240] Negotiation successful
0809 22:47:15.613595 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":445,"threads_started":1}
I20260809 22:47:15.616452 10095 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34237
I20260809 22:47:15.616901 10095 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:15.617584 10095 heartbeater.cc:511] Master 127.24.136.62:34237 requested a full tablet report, sending...
I20260809 22:47:15.620126 9783 ts_manager.cc:194] Registered new tserver with Master: b624d53d8a0f413ca1a3dcf3ec0fa193 (127.24.136.1:43839)
W20260809 22:47:15.965539 10101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:15.965878 10101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:15.966006 10101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:15.974291 10101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:15.974419 10101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:15.974565 10101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:47:15.984447 10101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:43229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:15.985877 10101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:15.987893 10101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:16.002282 10109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:16.002178 10110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:16.002301 10116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:16.003149 10101 server_base.cc:1034] running on GCE node
I20260809 22:47:16.003933 10101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:16.005088 10101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:16.006313 10101 hybrid_clock.cc:648] HybridClock initialized: now 1786315636006240 us; error 66 us; skew 500 ppm
I20260809 22:47:16.006657 10101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:47:16.009276 10101 webserver.cc:460] Webserver started at http://127.24.136.2:42141/ using document root <none> and password file <none>
I20260809 22:47:16.010042 10101 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:16.010146 10101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:16.010540 10101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:16.013746 10101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "1d022e4fc58b484aa6d3624574eb69ee"
format_stamp: "Formatted at 2026-08-09 22:47:16 on dist-test-slave-pww0"
server_key: "8106fa994a98b21419292b23594d122f"
server_key_iv: "454a29450a264c7b7031c3c32e692e79"
server_key_version: "encryptionkey@0"
I20260809 22:47:16.014611 10101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "1d022e4fc58b484aa6d3624574eb69ee"
format_stamp: "Formatted at 2026-08-09 22:47:16 on dist-test-slave-pww0"
server_key: "8106fa994a98b21419292b23594d122f"
server_key_iv: "454a29450a264c7b7031c3c32e692e79"
server_key_version: "encryptionkey@0"
I20260809 22:47:16.020303 10101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:47:16.024329 10124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:16.025897 10101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:47:16.026073 10101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "1d022e4fc58b484aa6d3624574eb69ee"
format_stamp: "Formatted at 2026-08-09 22:47:16 on dist-test-slave-pww0"
server_key: "8106fa994a98b21419292b23594d122f"
server_key_iv: "454a29450a264c7b7031c3c32e692e79"
server_key_version: "encryptionkey@0"
I20260809 22:47:16.026292 10101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:16.032429 10101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:16.033275 10101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:16.033529 10101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:16.034556 10101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:16.036125 10101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:16.036219 10101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:16.036334 10101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:16.036401 10101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:16.070170 10101 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:38793
I20260809 22:47:16.070228 10322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:38793 every 8 connection(s)
I20260809 22:47:16.071637 10101 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:47:16.073531 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 10101
I20260809 22:47:16.073722 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:47:16.074205 25120 external_mini_cluster.cc:1428] Setting key ab2cd0b360b2983e3303010973673805
I20260809 22:47:16.077229 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34237
--builtin_ntp_servers=127.24.136.20:43229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 22:47:16.080770 10099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:16.075731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:57181 (local address 127.24.136.62:34237)
0809 22:47:16.075922 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:16.075928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:16.076448 (+ 520us) server_negotiation.cc:407] Connection header received
0809 22:47:16.078394 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:16.078401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:16.078473 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:16.078585 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:16.079999 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:16.080005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:16.080009 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:16.080030 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:16.080077 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:16.080094 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:16.080207 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:16.080429 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 22:47:16.080498 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:47:16.081710 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:16.075140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34237 (local address 127.24.136.2:57181)
0809 22:47:16.076270 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:16.076321 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:16.078110 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:16.078784 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:16.078792 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:16.078892 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:16.079642 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:47:16.079645 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:16.079693 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:47:16.079792 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:16.080189 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:16.080196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:16.080202 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:47:16.080477 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 22:47:16.080707 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":565,"threads_started":1}
I20260809 22:47:16.083196 10323 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34237
I20260809 22:47:16.083572 10323 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:16.084297 10323 heartbeater.cc:511] Master 127.24.136.62:34237 requested a full tablet report, sending...
I20260809 22:47:16.086043 9783 ts_manager.cc:194] Registered new tserver with Master: 1d022e4fc58b484aa6d3624574eb69ee (127.24.136.2:38793)
W20260809 22:47:16.433997 10332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:16.434301 10332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:16.434414 10332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:16.441869 10332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:16.441994 10332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:16.442150 10332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:47:16.451854 10332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:43229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:34237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:16.453231 10332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:16.455171 10332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:16.469362 10339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:16.469327 10342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:16.469393 10338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:16.470113 10332 server_base.cc:1034] running on GCE node
I20260809 22:47:16.470806 10332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:16.471894 10332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:16.473151 10332 hybrid_clock.cc:648] HybridClock initialized: now 1786315636473087 us; error 55 us; skew 500 ppm
I20260809 22:47:16.473519 10332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:47:16.476172 10332 webserver.cc:460] Webserver started at http://127.24.136.3:41703/ using document root <none> and password file <none>
I20260809 22:47:16.476998 10332 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:16.477118 10332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:16.477519 10332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:16.480835 10332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "6196e1a7c7a6450cad0a2192ce8f0ca9"
format_stamp: "Formatted at 2026-08-09 22:47:16 on dist-test-slave-pww0"
server_key: "5bc5fed7ec093be8cd35c522d1728091"
server_key_iv: "71c8e5c91ea5230df69cbf8618d8bde6"
server_key_version: "encryptionkey@0"
I20260809 22:47:16.481817 10332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "6196e1a7c7a6450cad0a2192ce8f0ca9"
format_stamp: "Formatted at 2026-08-09 22:47:16 on dist-test-slave-pww0"
server_key: "5bc5fed7ec093be8cd35c522d1728091"
server_key_iv: "71c8e5c91ea5230df69cbf8618d8bde6"
server_key_version: "encryptionkey@0"
I20260809 22:47:16.487874 10332 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 22:47:16.491968 10354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:16.493688 10332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:47:16.493901 10332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "6196e1a7c7a6450cad0a2192ce8f0ca9"
format_stamp: "Formatted at 2026-08-09 22:47:16 on dist-test-slave-pww0"
server_key: "5bc5fed7ec093be8cd35c522d1728091"
server_key_iv: "71c8e5c91ea5230df69cbf8618d8bde6"
server_key_version: "encryptionkey@0"
I20260809 22:47:16.494151 10332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:16.500550 10332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:16.501468 10332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:16.501752 10332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:16.502792 10332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:16.504379 10332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:16.504485 10332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:16.504609 10332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:16.504683 10332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:16.536685 10332 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:46003
I20260809 22:47:16.536759 10551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:46003 every 8 connection(s)
I20260809 22:47:16.538035 10332 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:47:16.538935 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 10332
I20260809 22:47:16.539129 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:47:16.539705 25120 external_mini_cluster.cc:1428] Setting key 71efd4fdc62311c2e71fef08fb58aabb
I20260809 22:47:16.546864 10099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:16.540870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:54779 (local address 127.24.136.62:34237)
0809 22:47:16.541032 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:16.541039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:16.542828 (+ 1789us) server_negotiation.cc:407] Connection header received
0809 22:47:16.544633 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:16.544638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:16.544709 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:16.544798 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:16.546108 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:16.546114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:16.546120 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:16.546141 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:16.546173 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:16.546192 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:16.546300 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:16.546489 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 22:47:16.546561 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:47:16.547909 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:16.541544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:34237 (local address 127.24.136.3:54779)
0809 22:47:16.542659 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:16.542725 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:16.544353 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:16.545052 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:16.545061 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:16.545161 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:16.545806 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:47:16.545808 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:16.545849 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:47:16.545933 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:16.546285 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:16.546296 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:16.546304 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:47:16.546569 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 22:47:16.546869 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":316,"threads_started":1}
I20260809 22:47:16.549342 10553 heartbeater.cc:347] Connected to a master server at 127.24.136.62:34237
I20260809 22:47:16.549623 10553 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:16.550215 10553 heartbeater.cc:511] Master 127.24.136.62:34237 requested a full tablet report, sending...
I20260809 22:47:16.551715 9783 ts_manager.cc:194] Registered new tserver with Master: 6196e1a7c7a6450cad0a2192ce8f0ca9 (127.24.136.3:46003)
I20260809 22:47:16.555612 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 22:47:16.561972 10099 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:47:16.560423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55092 (local address 127.24.136.62:34237)
0809 22:47:16.560672 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:16.560679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:16.561033 (+ 354us) server_negotiation.cc:407] Connection header received
0809 22:47:16.561198 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:16.561202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:16.561263 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:16.561348 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:16.561717 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55092: BlockingRecv error: recv got EOF from 127.0.0.1:55092 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:47:16.562605 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 9880
I20260809 22:47:16.566313 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:16.563598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.24.136.1:43839)
0809 22:47:16.564166 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:16.564177 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:16.564204 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:47:16.564265 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:16.564272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:16.564515 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:16.564669 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:16.565611 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:16.565618 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:16.565630 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:16.565643 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:16.565691 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:16.565709 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:16.565827 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:16.565940 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 22:47:16.566050 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":257,"threads_started":1}
I20260809 22:47:16.567636 10084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:16.648638 10095 heartbeater.cc:503] Master 127.24.136.62:34237 was elected leader, sending a full tablet report...
I20260809 22:47:16.649847 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 9880
W20260809 22:47:16.671857 9759 connection.cc:535] server connection from 127.24.136.1:36785 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 22:47:16.672042 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 10101
I20260809 22:47:16.676070 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:16.673161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.24.136.2:38793)
0809 22:47:16.673744 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:16.673761 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:16.673792 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:47:16.673859 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:16.673865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:16.674084 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:16.674216 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:16.675107 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:16.675115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:16.675130 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:16.675150 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:16.675197 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:16.675213 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:16.675336 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:16.675697 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 22:47:16.675813 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":264,"threads_started":1}
I20260809 22:47:16.677424 10314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:16.759617 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 10101
I20260809 22:47:16.781973 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 10332
I20260809 22:47:16.786186 10572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:16.783003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33170 (local address 127.24.136.3:46003)
0809 22:47:16.783700 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:16.783708 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:16.783743 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:47:16.783811 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:16.783818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:16.784042 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:16.784198 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:16.785377 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:16.785384 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:16.785397 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:47:16.785411 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:16.785457 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:47:16.785470 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:16.785647 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:16.785735 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 22:47:16.785855 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":316,"threads_started":1}
I20260809 22:47:16.787568 10547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:16.870177 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 10332
I20260809 22:47:16.889135 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 9734
I20260809 22:47:16.890439 9827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:16.994519 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 9734
2026-08-09T22:47:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2494 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:47:17 dist-test-slave-pww0 krb5kdc[10579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:47:17 dist-test-slave-pww0 krb5kdc[10579](info): set up 2 sockets
Aug 09 22:47:17 dist-test-slave-pww0 krb5kdc[10579](info): commencing operation
krb5kdc: starting...
W20260809 22:47:18.277268 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:47:18 dist-test-slave-pww0 krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:47:18.570605 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:44491
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:44491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:47:18.926965 10599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:18.927307 10599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:18.927434 10599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:18.935171 10599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:47:18.935307 10599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:18.935365 10599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:18.935478 10599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:47:18.935547 10599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:47:18.945956 10599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:44491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:44491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:18.947417 10599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:18.948854 10599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:18.961356 10609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:18.961354 10610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:18.961359 10613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:18.962240 10599 server_base.cc:1034] running on GCE node
I20260809 22:47:18.963173 10599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:18.964834 10599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:18.966049 10599 hybrid_clock.cc:648] HybridClock initialized: now 1786315638965989 us; error 51 us; skew 500 ppm
I20260809 22:47:18.966428 10599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:18 dist-test-slave-pww0 krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:18.973917 10599 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:47:18.975688 10599 webserver.cc:460] Webserver started at http://127.24.136.62:46663/ using document root <none> and password file <none>
I20260809 22:47:18.976485 10599 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:18.976626 10599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:18.976960 10599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:18.980363 10599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "7950e87f39e14cab9cd9c9e89c2db419"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-pww0"
server_key: "6d61544606fa86576308208a50c400fa"
server_key_iv: "cb6a2526380f3c11fa869dbb41ccb4ab"
server_key_version: "encryptionkey@0"
I20260809 22:47:18.981201 10599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "7950e87f39e14cab9cd9c9e89c2db419"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-pww0"
server_key: "6d61544606fa86576308208a50c400fa"
server_key_iv: "cb6a2526380f3c11fa869dbb41ccb4ab"
server_key_version: "encryptionkey@0"
I20260809 22:47:18.986970 10599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260809 22:47:18.991006 10623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:18.992568 10599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:47:18.992772 10599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "7950e87f39e14cab9cd9c9e89c2db419"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-pww0"
server_key: "6d61544606fa86576308208a50c400fa"
server_key_iv: "cb6a2526380f3c11fa869dbb41ccb4ab"
server_key_version: "encryptionkey@0"
I20260809 22:47:18.992982 10599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:19.012708 10599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:19.017560 10599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:19.017870 10599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:19.034986 10599 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:44491
I20260809 22:47:19.035005 10717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:44491 every 8 connection(s)
I20260809 22:47:19.036485 10599 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:47:19.040640 10718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:19.041217 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 10599
I20260809 22:47:19.041432 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:47:19.042093 25120 external_mini_cluster.cc:1428] Setting key 474b7e6c2cd0ac7d49220aa07aee2ad0
I20260809 22:47:19.055807 10718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419: Bootstrap starting.
Aug 09 22:47:19 dist-test-slave-pww0 krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:47:19.060156 10718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:19.061542 10718 log.cc:824] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:19.065585 10718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419: No bootstrap required, opened a new log
I20260809 22:47:19.068497 10722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.044228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.24.136.62:44491)
0809 22:47:19.045120 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:19.045136 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:19.045189 (+ 53us) server_negotiation.cc:407] Connection header received
0809 22:47:19.047097 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:19.047129 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:19.047663 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:19.048071 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:19.050080 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.051503 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.053449 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.054292 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.054463 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.059671 (+ 5208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:19.059681 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:19.059699 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:19.059765 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:19.063495 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.064552 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.064561 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.064568 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.064805 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.065347 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.065352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.065354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.066058 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:19.066326 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.066847 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.067264 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":368,"threads_started":1}
I20260809 22:47:19.074753 10718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7950e87f39e14cab9cd9c9e89c2db419" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44491 } }
I20260809 22:47:19.075274 10718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:19.075394 10718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7950e87f39e14cab9cd9c9e89c2db419, State: Initialized, Role: FOLLOWER
I20260809 22:47:19.076226 10718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7950e87f39e14cab9cd9c9e89c2db419" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44491 } }
I20260809 22:47:19.076508 10718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:47:19.076658 10718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:47:19.076865 10718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:19.079839 10718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7950e87f39e14cab9cd9c9e89c2db419" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44491 } }
I20260809 22:47:19.080471 10718 leader_election.cc:302] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7950e87f39e14cab9cd9c9e89c2db419; no voters:
I20260809 22:47:19.081362 10718 leader_election.cc:288] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:47:19.081672 10725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:47:19.084034 10718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:47:19.088625 10725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [term 1 LEADER]: Becoming Leader. State: Replica: 7950e87f39e14cab9cd9c9e89c2db419, State: Running, Role: LEADER
I20260809 22:47:19.091238 10725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7950e87f39e14cab9cd9c9e89c2db419" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44491 } }
I20260809 22:47:19.095062 10729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7950e87f39e14cab9cd9c9e89c2db419. Latest consensus state: current_term: 1 leader_uuid: "7950e87f39e14cab9cd9c9e89c2db419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7950e87f39e14cab9cd9c9e89c2db419" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44491 } } }
I20260809 22:47:19.095600 10729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [sys.catalog]: This master's current role is: LEADER
I20260809 22:47:19.096148 10727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7950e87f39e14cab9cd9c9e89c2db419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7950e87f39e14cab9cd9c9e89c2db419" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44491 } } }
I20260809 22:47:19.096524 10727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419 [sys.catalog]: This master's current role is: LEADER
W20260809 22:47:19.107877 10748 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:47:19.108016 10748 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:47:19.108167 10743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:47:19.109876 10743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:47:19.119391 10743 catalog_manager.cc:1347] Generated new cluster ID: 0dadc07e82b04584aa9a9ac8e2066897
I20260809 22:47:19.119557 10743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:47:19.149679 10743 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:47:19.151419 10743 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:47:19.159825 10743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419: Generated new TSK 0
I20260809 22:47:19.160813 10743 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:47:19.245121 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:44491
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:47:19.606367 10758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:19.606690 10758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:19.606827 10758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:19.614454 10758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:19.614589 10758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:19.614763 10758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:47:19.625804 10758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:44491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:19.627290 10758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:19.628808 10758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:19.643775 10767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:19.643854 10766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:19.644021 10769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:19.644618 10758 server_base.cc:1034] running on GCE node
I20260809 22:47:19.645361 10758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:19.646462 10758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:19.647689 10758 hybrid_clock.cc:648] HybridClock initialized: now 1786315639647629 us; error 51 us; skew 500 ppm
I20260809 22:47:19.648041 10758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:19 dist-test-slave-pww0 krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315639, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:19.651935 10758 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:47:19.653749 10758 webserver.cc:460] Webserver started at http://127.24.136.1:46409/ using document root <none> and password file <none>
I20260809 22:47:19.654541 10758 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:19.654671 10758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:19.655001 10758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:19.658310 10758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "01f9685bbe09441f99e35aa43c863406"
format_stamp: "Formatted at 2026-08-09 22:47:19 on dist-test-slave-pww0"
server_key: "def9ffbebfcb6bfbfbe059a305ee0f8c"
server_key_iv: "6ce75f904f468a0f43b9b1fcfefe1e97"
server_key_version: "encryptionkey@0"
I20260809 22:47:19.659200 10758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "01f9685bbe09441f99e35aa43c863406"
format_stamp: "Formatted at 2026-08-09 22:47:19 on dist-test-slave-pww0"
server_key: "def9ffbebfcb6bfbfbe059a305ee0f8c"
server_key_iv: "6ce75f904f468a0f43b9b1fcfefe1e97"
server_key_version: "encryptionkey@0"
I20260809 22:47:19.665292 10758 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 22:47:19.669396 10779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:19.671000 10758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:47:19.671200 10758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "01f9685bbe09441f99e35aa43c863406"
format_stamp: "Formatted at 2026-08-09 22:47:19 on dist-test-slave-pww0"
server_key: "def9ffbebfcb6bfbfbe059a305ee0f8c"
server_key_iv: "6ce75f904f468a0f43b9b1fcfefe1e97"
server_key_version: "encryptionkey@0"
I20260809 22:47:19.671420 10758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:19.698730 10758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:19.703301 10758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:19.703647 10758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:19.704699 10758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:19.706136 10758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:19.706221 10758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:19.706346 10758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:19.706400 10758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:19.739014 10758 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:39429
I20260809 22:47:19.739063 10971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:39429 every 8 connection(s)
I20260809 22:47:19.740434 10758 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:47:19.746604 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 10758
I20260809 22:47:19.746869 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:47:19.747752 25120 external_mini_cluster.cc:1428] Setting key f4d3d59495e141d1d1ca73892fc425a6
Aug 09 22:47:19 dist-test-slave-pww0 krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315639, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:47:19.762893 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.743254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:49883 (local address 127.24.136.62:44491)
0809 22:47:19.743793 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:19.743798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:19.744858 (+ 1060us) server_negotiation.cc:407] Connection header received
0809 22:47:19.747024 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:19.747029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:19.747108 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:19.747209 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:19.749306 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.750078 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.751782 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.752216 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.752334 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.756156 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:19.756162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:19.756165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:19.756203 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:19.759361 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.760331 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.760337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.760340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.760509 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.760960 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.760965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.760968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.761326 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:19.761478 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.762413 (+ 935us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.762618 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":297,"threads_started":1}
I20260809 22:47:19.764434 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.743898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:44491 (local address 127.24.136.1:49883)
0809 22:47:19.744677 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:19.744718 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:19.746629 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:19.747588 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:19.747603 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:19.748402 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:19.749104 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:19.749125 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.750252 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.750256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.751621 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:19.751642 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.752460 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.752468 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.752719 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.753394 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:19.753420 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:19.755885 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:19.759614 (+ 3729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:19.759624 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:19.759641 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:19.760154 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:19.760616 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:19.760619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:19.760621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:19.760843 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:19.761475 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:19.761484 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:19.761980 (+ 496us) client_negotiation.cc:769] Sending connection context
0809 22:47:19.762317 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 22:47:19.762938 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":289,"threads_started":1}
I20260809 22:47:19.766299 10973 heartbeater.cc:347] Connected to a master server at 127.24.136.62:44491
I20260809 22:47:19.766686 10973 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:19.767640 10973 heartbeater.cc:511] Master 127.24.136.62:44491 requested a full tablet report, sending...
I20260809 22:47:19.770161 10656 ts_manager.cc:194] Registered new tserver with Master: 01f9685bbe09441f99e35aa43c863406 (127.24.136.1:39429)
I20260809 22:47:19.771793 10656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:49883
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:47:19.815744 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:44491
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:47:20.173168 10980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:20.173533 10980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:20.173676 10980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:20.181429 10980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:20.181587 10980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:20.181744 10980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:47:20.191933 10980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:44491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:20.193416 10980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:20.194895 10980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:20.209183 10990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:20.209113 10991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:20.209136 10995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:20.210011 10980 server_base.cc:1034] running on GCE node
I20260809 22:47:20.210724 10980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:20.211848 10980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:20.213104 10980 hybrid_clock.cc:648] HybridClock initialized: now 1786315640213027 us; error 67 us; skew 500 ppm
I20260809 22:47:20.213497 10980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:20 dist-test-slave-pww0 krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:20.217154 10980 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:47:20.219246 10980 webserver.cc:460] Webserver started at http://127.24.136.2:38739/ using document root <none> and password file <none>
I20260809 22:47:20.220115 10980 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:20.220258 10980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:20.220600 10980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:20.224026 10980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "a9395a9ebb3e448190225f1484bfb835"
format_stamp: "Formatted at 2026-08-09 22:47:20 on dist-test-slave-pww0"
server_key: "03dd28f4af069861dff2b16b41fd33ac"
server_key_iv: "c4647de02ab02b9f572c1dcd888bd3b7"
server_key_version: "encryptionkey@0"
I20260809 22:47:20.224952 10980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "a9395a9ebb3e448190225f1484bfb835"
format_stamp: "Formatted at 2026-08-09 22:47:20 on dist-test-slave-pww0"
server_key: "03dd28f4af069861dff2b16b41fd33ac"
server_key_iv: "c4647de02ab02b9f572c1dcd888bd3b7"
server_key_version: "encryptionkey@0"
I20260809 22:47:20.230782 10980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:47:20.234889 11009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:20.236479 10980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:47:20.236676 10980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "a9395a9ebb3e448190225f1484bfb835"
format_stamp: "Formatted at 2026-08-09 22:47:20 on dist-test-slave-pww0"
server_key: "03dd28f4af069861dff2b16b41fd33ac"
server_key_iv: "c4647de02ab02b9f572c1dcd888bd3b7"
server_key_version: "encryptionkey@0"
I20260809 22:47:20.236893 10980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:20.257587 10980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:20.262455 10980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:20.262758 10980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:20.263825 10980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:20.265452 10980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:20.265537 10980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:20.265645 10980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:20.265695 10980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:20.297868 10980 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:40435
I20260809 22:47:20.297884 11226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:40435 every 8 connection(s)
I20260809 22:47:20.299257 10980 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:47:20.306402 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 10980
I20260809 22:47:20.306594 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:47:20.307024 25120 external_mini_cluster.cc:1428] Setting key 29f702de852cb24bf5d89b416bd71986
Aug 09 22:47:20 dist-test-slave-pww0 krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:47:20.323273 11231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.302169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:52919 (local address 127.24.136.62:44491)
0809 22:47:20.302635 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:20.302639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:20.303561 (+ 922us) server_negotiation.cc:407] Connection header received
0809 22:47:20.305776 (+ 2215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:20.305783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:20.305877 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:20.305981 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:20.307943 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.308805 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.310653 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.311367 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.311597 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.315706 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:20.315716 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:20.315720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:20.315761 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:20.319009 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:20.320325 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:20.320331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:20.320333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:20.320504 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:20.321100 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:20.321105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:20.321107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:20.321446 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:20.321610 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:20.322593 (+ 983us) server_negotiation.cc:299] Negotiation successful
0809 22:47:20.322882 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":247,"threads_started":1}
I20260809 22:47:20.324050 11232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.302609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:44491 (local address 127.24.136.2:52919)
0809 22:47:20.303354 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:20.303398 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:20.305440 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:20.306280 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:20.306289 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:20.306942 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:20.307716 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.307746 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.308997 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.309003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.310450 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.310465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.311700 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.311706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.311922 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.312720 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:20.312756 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:20.315332 (+ 2576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:20.319309 (+ 3977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:20.319322 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:20.319345 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:20.320110 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:20.320634 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:20.320639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:20.320642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:20.320937 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:20.321601 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:20.321610 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:20.322135 (+ 525us) client_negotiation.cc:769] Sending connection context
0809 22:47:20.322408 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 22:47:20.322862 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":274,"threads_started":1}
I20260809 22:47:20.325680 11227 heartbeater.cc:347] Connected to a master server at 127.24.136.62:44491
I20260809 22:47:20.326062 11227 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:20.327080 11227 heartbeater.cc:511] Master 127.24.136.62:44491 requested a full tablet report, sending...
I20260809 22:47:20.329103 10656 ts_manager.cc:194] Registered new tserver with Master: a9395a9ebb3e448190225f1484bfb835 (127.24.136.2:40435)
I20260809 22:47:20.329999 10656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:52919
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:47:20.380275 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:44491
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:47:20.736632 11238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:20.736951 11238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:20.737084 11238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:20.745007 11238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:20.745138 11238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:20.745289 11238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:47:20.755659 11238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:44491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:47:20.757077 11238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:20.758495 11238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:20.772388 11252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:20.772423 11255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:20.772339 11249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:20.773483 11238 server_base.cc:1034] running on GCE node
I20260809 22:47:20.774199 11238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:20.775224 11238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:20.775875 10973 heartbeater.cc:503] Master 127.24.136.62:44491 was elected leader, sending a full tablet report...
I20260809 22:47:20.776439 11238 hybrid_clock.cc:648] HybridClock initialized: now 1786315640776376 us; error 56 us; skew 500 ppm
I20260809 22:47:20.776822 11238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:20 dist-test-slave-pww0 krb5kdc[10579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:20.780371 11238 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:47:20.781910 11238 webserver.cc:460] Webserver started at http://127.24.136.3:37107/ using document root <none> and password file <none>
I20260809 22:47:20.782660 11238 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:20.782784 11238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:20.783206 11238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:20.787192 11238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "f683e6fc72a84154aa4843956331cee0"
format_stamp: "Formatted at 2026-08-09 22:47:20 on dist-test-slave-pww0"
server_key: "4834f727ce010d20c054daefe33f6f48"
server_key_iv: "2a918d539a426de1e60c446ca48bdfd4"
server_key_version: "encryptionkey@0"
I20260809 22:47:20.788136 11238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "f683e6fc72a84154aa4843956331cee0"
format_stamp: "Formatted at 2026-08-09 22:47:20 on dist-test-slave-pww0"
server_key: "4834f727ce010d20c054daefe33f6f48"
server_key_iv: "2a918d539a426de1e60c446ca48bdfd4"
server_key_version: "encryptionkey@0"
I20260809 22:47:20.794268 11238 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260809 22:47:20.798256 11269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:20.799739 11238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:47:20.799922 11238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "f683e6fc72a84154aa4843956331cee0"
format_stamp: "Formatted at 2026-08-09 22:47:20 on dist-test-slave-pww0"
server_key: "4834f727ce010d20c054daefe33f6f48"
server_key_iv: "2a918d539a426de1e60c446ca48bdfd4"
server_key_version: "encryptionkey@0"
I20260809 22:47:20.800153 11238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:20.816193 11238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:20.821120 11238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:20.821444 11238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:20.822510 11238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:20.824093 11238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:20.824188 11238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:20.824316 11238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:20.824375 11238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:20.857645 11238 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:40999
I20260809 22:47:20.857729 11477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:40999 every 8 connection(s)
I20260809 22:47:20.859028 11238 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:47:20.861847 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 11238
I20260809 22:47:20.862071 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:47:20.862608 25120 external_mini_cluster.cc:1428] Setting key 621edd0de42b270aea7ef0c5c9154562
Aug 09 22:47:20 dist-test-slave-pww0 krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315640, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:47:20.881754 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.861973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:40945 (local address 127.24.136.62:44491)
0809 22:47:20.862575 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:20.862582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:20.863774 (+ 1192us) server_negotiation.cc:407] Connection header received
0809 22:47:20.865809 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:20.865816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:20.865902 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:20.865997 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:20.867830 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.868626 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.870356 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.870798 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.870896 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.874822 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:20.874828 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:20.874832 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:20.874873 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:20.878285 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:20.879273 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:20.879277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:20.879280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:20.879433 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:20.880032 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:20.880036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:20.880038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:20.880335 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:20.880454 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:20.881249 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 22:47:20.881453 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":283,"threads_started":1}
I20260809 22:47:20.882682 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.862571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:44491 (local address 127.24.136.3:40945)
0809 22:47:20.863583 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:20.863646 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:20.865514 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:20.866259 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:20.866270 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:20.866887 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:20.867620 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.867646 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.868820 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.868825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.870187 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.870201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.871034 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.871040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.871270 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.872032 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:20.872061 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:20.874495 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:20.878474 (+ 3979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:20.878487 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:20.878508 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:20.879099 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:20.879624 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:20.879628 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:20.879631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:20.879898 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:20.880457 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:20.880464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:20.880854 (+ 390us) client_negotiation.cc:769] Sending connection context
0809 22:47:20.881106 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 22:47:20.881549 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":368,"threads_started":1}
I20260809 22:47:20.884265 11478 heartbeater.cc:347] Connected to a master server at 127.24.136.62:44491
I20260809 22:47:20.884610 11478 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:20.885413 11478 heartbeater.cc:511] Master 127.24.136.62:44491 requested a full tablet report, sending...
I20260809 22:47:20.887055 10656 ts_manager.cc:194] Registered new tserver with Master: f683e6fc72a84154aa4843956331cee0 (127.24.136.3:40999)
I20260809 22:47:20.887961 10656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:40945
I20260809 22:47:20.891247 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:47:20.909199 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.895884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56526 (local address 127.24.136.62:44491)
0809 22:47:20.896112 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:20.896121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:20.896528 (+ 407us) server_negotiation.cc:407] Connection header received
0809 22:47:20.896684 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:20.896688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:20.896760 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:20.896842 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:20.898414 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.899165 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.900547 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.901009 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.901124 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.903490 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:20.903499 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:20.903504 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:20.903544 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:20.906387 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:20.907179 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:20.907185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:20.907188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:20.907343 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:20.907792 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:20.907796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:20.907798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:20.908166 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:20.908381 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:20.908748 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 22:47:20.908951 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:47:20.913858 10656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:47:20.921190 10656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:47:20.958081 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.942851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47618 (local address 127.24.136.2:40435)
0809 22:47:20.950501 (+ 7650us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:20.950509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:20.950535 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:47:20.950637 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:20.950645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:20.950854 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:20.951074 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:20.951711 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.953283 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.955803 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.956848 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.956990 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.957206 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:20.957587 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 22:47:20.957773 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7503,"thread_start_us":7348,"threads_started":1}
I20260809 22:47:20.958740 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.935955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.24.136.3:40999)
0809 22:47:20.948325 (+ 12370us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:20.948337 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:20.949757 (+ 1420us) server_negotiation.cc:407] Connection header received
0809 22:47:20.949875 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:20.949883 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:20.950108 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:20.950335 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:20.951138 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.952824 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.955137 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.956536 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.956681 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.956945 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:20.958204 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0809 22:47:20.958386 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12133,"thread_start_us":12396,"threads_started":1}
I20260809 22:47:20.958796 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.935094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.24.136.1:39429)
0809 22:47:20.949083 (+ 13989us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:20.949094 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:20.949129 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:47:20.949261 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:20.949270 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:20.949581 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:20.950625 (+ 1044us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:20.950834 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.952487 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.955614 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.957080 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.958003 (+ 923us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.958283 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:20.958358 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 22:47:20.958510 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13806,"thread_start_us":13641,"threads_started":1}
I20260809 22:47:20.957903 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.935399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:40999 (local address 127.0.0.1:41880)
0809 22:47:20.949414 (+ 14015us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:20.949434 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:20.949599 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:20.950509 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:20.950514 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:20.950541 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:20.950931 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.950944 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.953010 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.953015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.954961 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.954974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.956767 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.956775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.957004 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.957074 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 22:47:20.957222 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 22:47:20.957420 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13872,"mutex_wait_us":146,"thread_start_us":13702,"threads_started":1}
I20260809 22:47:20.959223 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.934927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39429 (local address 127.0.0.1:56780)
0809 22:47:20.941900 (+ 6973us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:20.941953 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:20.942165 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:20.949865 (+ 7700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:20.949873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:20.949907 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:20.950377 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.950394 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.952652 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.952657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.955402 (+ 2745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.955419 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.957332 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.957339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.957527 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.957560 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:47:20.957611 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 22:47:20.957721 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6685,"mutex_wait_us":47,"spinlock_wait_cycles":768,"thread_start_us":6538,"threads_started":1}
I20260809 22:47:20.958086 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:20.947794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:40435 (local address 127.0.0.1:47618)
0809 22:47:20.949799 (+ 2005us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:20.949819 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:20.949961 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:20.951071 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:20.951075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:20.951100 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:20.951445 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.951483 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.953445 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.953450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.955650 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:20.955662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:20.957127 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:20.957134 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:20.957319 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:20.957407 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 22:47:20.957487 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 22:47:20.957627 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1869,"mutex_wait_us":46,"thread_start_us":1701,"threads_started":1}
I20260809 22:47:20.963536 11097 tablet_service.cc:1467] Processing CreateTablet for tablet f53a950fdbe946a1a1aff3029d3abbac (DEFAULT_TABLE table=test-table [id=3715b9bee64d4827867bbc58f320a24c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:47:20.963310 11358 tablet_service.cc:1467] Processing CreateTablet for tablet f53a950fdbe946a1a1aff3029d3abbac (DEFAULT_TABLE table=test-table [id=3715b9bee64d4827867bbc58f320a24c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:47:20.963310 10870 tablet_service.cc:1467] Processing CreateTablet for tablet f53a950fdbe946a1a1aff3029d3abbac (DEFAULT_TABLE table=test-table [id=3715b9bee64d4827867bbc58f320a24c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:47:20.964870 11097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53a950fdbe946a1a1aff3029d3abbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:20.964974 10870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53a950fdbe946a1a1aff3029d3abbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:20.964991 11358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53a950fdbe946a1a1aff3029d3abbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:20.978269 11517 tablet_bootstrap.cc:492] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: Bootstrap starting.
I20260809 22:47:20.980697 11518 tablet_bootstrap.cc:492] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: Bootstrap starting.
I20260809 22:47:20.982496 11519 tablet_bootstrap.cc:492] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: Bootstrap starting.
I20260809 22:47:20.982534 11517 tablet_bootstrap.cc:654] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:20.983989 11517 log.cc:824] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:20.985020 11518 tablet_bootstrap.cc:654] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:20.985857 11519 tablet_bootstrap.cc:654] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:20.986517 11518 log.cc:824] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:20.986886 11519 log.cc:824] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:20.993417 11519 tablet_bootstrap.cc:492] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: No bootstrap required, opened a new log
I20260809 22:47:20.993461 11517 tablet_bootstrap.cc:492] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: No bootstrap required, opened a new log
I20260809 22:47:20.993462 11518 tablet_bootstrap.cc:492] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: No bootstrap required, opened a new log
I20260809 22:47:20.993786 11519 ts_tablet_manager.cc:1397] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260809 22:47:20.993796 11518 ts_tablet_manager.cc:1397] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260809 22:47:20.993805 11517 ts_tablet_manager.cc:1397] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.005s
I20260809 22:47:21.001991 11519 raft_consensus.cc:348] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } }
I20260809 22:47:21.002439 11519 raft_consensus.cc:374] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:21.002562 11519 raft_consensus.cc:729] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f683e6fc72a84154aa4843956331cee0, State: Initialized, Role: FOLLOWER
I20260809 22:47:21.003227 11519 consensus_queue.cc:260] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } }
I20260809 22:47:21.004518 11478 heartbeater.cc:503] Master 127.24.136.62:44491 was elected leader, sending a full tablet report...
I20260809 22:47:21.004658 11517 raft_consensus.cc:348] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } }
I20260809 22:47:21.005147 11517 raft_consensus.cc:374] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:21.005247 11517 raft_consensus.cc:729] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9395a9ebb3e448190225f1484bfb835, State: Initialized, Role: FOLLOWER
I20260809 22:47:21.004951 11518 raft_consensus.cc:348] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } }
I20260809 22:47:21.005369 11519 ts_tablet_manager.cc:1428] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260809 22:47:21.005398 11518 raft_consensus.cc:374] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:21.005486 11518 raft_consensus.cc:729] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01f9685bbe09441f99e35aa43c863406, State: Initialized, Role: FOLLOWER
I20260809 22:47:21.006042 11517 consensus_queue.cc:260] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } }
I20260809 22:47:21.006122 11518 consensus_queue.cc:260] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } }
I20260809 22:47:21.008049 11517 ts_tablet_manager.cc:1428] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260809 22:47:21.008301 11518 ts_tablet_manager.cc:1428] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: Time spent starting tablet: real 0.014s user 0.004s sys 0.011s
I20260809 22:47:21.009248 11227 heartbeater.cc:503] Master 127.24.136.62:44491 was elected leader, sending a full tablet report...
W20260809 22:47:21.051532 11229 tablet.cc:2369] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:47:21.110970 11479 tablet.cc:2369] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:47:21.154798 11534 raft_consensus.cc:482] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:47:21.155220 11534 raft_consensus.cc:504] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } }
I20260809 22:47:21.157193 11534 leader_election.cc:288] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9395a9ebb3e448190225f1484bfb835 (127.24.136.2:40435), f683e6fc72a84154aa4843956331cee0 (127.24.136.3:40999)
I20260809 22:47:21.164381 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.157631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:33251 (local address 127.24.136.2:40435)
0809 22:47:21.157877 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:21.157884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:21.158665 (+ 781us) server_negotiation.cc:407] Connection header received
0809 22:47:21.158826 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:21.158829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:21.158892 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:21.158987 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:21.159632 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.160508 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.162474 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.163290 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.163491 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:21.163571 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:21.163974 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 22:47:21.164109 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 22:47:21.164381 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.157850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:33477 (local address 127.24.136.3:40999)
0809 22:47:21.158059 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:21.158066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:21.158523 (+ 457us) server_negotiation.cc:407] Connection header received
0809 22:47:21.158753 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:21.158759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:21.158840 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:21.158908 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:21.159589 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.160402 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.162348 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.163229 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.163375 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:21.163482 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:21.163973 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 22:47:21.164117 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:47:21.164700 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.157528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:40999 (local address 127.24.136.1:33477)
0809 22:47:21.158395 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.158418 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:21.158584 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:21.158949 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:21.158954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:21.158981 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:21.159414 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:21.159428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.160530 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.160535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:21.162140 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:21.162158 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.163537 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.163546 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:21.163799 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:21.163829 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:47:21.163921 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 22:47:21.164088 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":50,"thread_start_us":750,"threads_started":1}
I20260809 22:47:21.164810 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.157591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:40435 (local address 127.24.136.1:33251)
0809 22:47:21.158559 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.158573 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:21.158707 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:21.159022 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:21.159027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:21.159056 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:21.159444 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:21.159481 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.160659 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.160663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:21.162263 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:21.162282 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.163576 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.163581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:21.163809 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:21.163835 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:47:21.163923 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:47:21.164097 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"mutex_wait_us":85,"thread_start_us":442,"threads_started":1}
I20260809 22:47:21.165773 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53a950fdbe946a1a1aff3029d3abbac" candidate_uuid: "01f9685bbe09441f99e35aa43c863406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9395a9ebb3e448190225f1484bfb835" is_pre_election: true
I20260809 22:47:21.165766 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53a950fdbe946a1a1aff3029d3abbac" candidate_uuid: "01f9685bbe09441f99e35aa43c863406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f683e6fc72a84154aa4843956331cee0" is_pre_election: true
I20260809 22:47:21.166280 11398 raft_consensus.cc:2454] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01f9685bbe09441f99e35aa43c863406 in term 0.
I20260809 22:47:21.166280 11135 raft_consensus.cc:2454] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01f9685bbe09441f99e35aa43c863406 in term 0.
I20260809 22:47:21.166960 10782 leader_election.cc:302] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01f9685bbe09441f99e35aa43c863406, f683e6fc72a84154aa4843956331cee0; no voters:
I20260809 22:47:21.167392 11534 raft_consensus.cc:2781] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:47:21.167572 11534 raft_consensus.cc:482] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:47:21.167693 11534 raft_consensus.cc:3037] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:21.169970 11534 raft_consensus.cc:504] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } }
I20260809 22:47:21.170656 11534 leader_election.cc:288] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [CANDIDATE]: Term 1 election: Requested vote from peers a9395a9ebb3e448190225f1484bfb835 (127.24.136.2:40435), f683e6fc72a84154aa4843956331cee0 (127.24.136.3:40999)
I20260809 22:47:21.171272 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53a950fdbe946a1a1aff3029d3abbac" candidate_uuid: "01f9685bbe09441f99e35aa43c863406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f683e6fc72a84154aa4843956331cee0"
I20260809 22:47:21.171272 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53a950fdbe946a1a1aff3029d3abbac" candidate_uuid: "01f9685bbe09441f99e35aa43c863406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9395a9ebb3e448190225f1484bfb835"
I20260809 22:47:21.171535 11398 raft_consensus.cc:3037] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:21.171535 11135 raft_consensus.cc:3037] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:21.174885 11398 raft_consensus.cc:2454] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01f9685bbe09441f99e35aa43c863406 in term 1.
I20260809 22:47:21.174885 11135 raft_consensus.cc:2454] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01f9685bbe09441f99e35aa43c863406 in term 1.
I20260809 22:47:21.175510 10782 leader_election.cc:302] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 01f9685bbe09441f99e35aa43c863406, a9395a9ebb3e448190225f1484bfb835, f683e6fc72a84154aa4843956331cee0; no voters:
I20260809 22:47:21.175902 11534 raft_consensus.cc:2781] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:47:21.176417 11534 raft_consensus.cc:686] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 1 LEADER]: Becoming Leader. State: Replica: 01f9685bbe09441f99e35aa43c863406, State: Running, Role: LEADER
I20260809 22:47:21.177073 11534 consensus_queue.cc:237] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } }
I20260809 22:47:21.182008 10656 catalog_manager.cc:5495] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01f9685bbe09441f99e35aa43c863406 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "01f9685bbe09441f99e35aa43c863406" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01f9685bbe09441f99e35aa43c863406" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 } health_report { overall_health: UNKNOWN } } }
W20260809 22:47:21.243126 10974 tablet.cc:2369] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:47:21.248052 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.242119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.24.136.1:39429)
0809 22:47:21.242368 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:21.242379 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:21.242525 (+ 146us) server_negotiation.cc:407] Connection header received
0809 22:47:21.242699 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:21.242703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:21.242845 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:21.243014 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:47:21.243450 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.244388 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.245782 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.246207 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.246331 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:21.246881 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:47:21.247413 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:47:21.247620 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:21.247673 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 22:47:21.247779 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 22:47:21.257519 11135 raft_consensus.cc:1261] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [term 1 FOLLOWER]: Refusing update from remote peer 01f9685bbe09441f99e35aa43c863406: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:47:21.257519 11398 raft_consensus.cc:1261] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [term 1 FOLLOWER]: Refusing update from remote peer 01f9685bbe09441f99e35aa43c863406: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:47:21.259004 11538 consensus_queue.cc:1035] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f683e6fc72a84154aa4843956331cee0" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:47:21.259389 11534 consensus_queue.cc:1035] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9395a9ebb3e448190225f1484bfb835" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 40435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:47:21.279801 11543 mvcc.cc:205] Tried to move back new op lower bound from 7316748866578530304 to 7316748866265378816. Current Snapshot: MvccSnapshot[applied={T|T < 7316748866578530304}]
I20260809 22:47:21.282037 11544 mvcc.cc:205] Tried to move back new op lower bound from 7316748866578530304 to 7316748866265378816. Current Snapshot: MvccSnapshot[applied={T|T < 7316748866578530304}]
I20260809 22:47:21.286636 11547 mvcc.cc:205] Tried to move back new op lower bound from 7316748866578530304 to 7316748866265378816. Current Snapshot: MvccSnapshot[applied={T|T < 7316748866578530304}]
I20260809 22:47:21.300381 10656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:47:21.300702 10656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:47:21.305348 10656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7950e87f39e14cab9cd9c9e89c2db419: Sending DeleteTablet for 3 replicas of tablet f53a950fdbe946a1a1aff3029d3abbac
I20260809 22:47:21.307080 11097 tablet_service.cc:1514] Processing DeleteTablet for tablet f53a950fdbe946a1a1aff3029d3abbac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:47:21 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:47618
I20260809 22:47:21.307063 11358 tablet_service.cc:1514] Processing DeleteTablet for tablet f53a950fdbe946a1a1aff3029d3abbac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:47:21 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:41880
I20260809 22:47:21.307869 10870 tablet_service.cc:1514] Processing DeleteTablet for tablet f53a950fdbe946a1a1aff3029d3abbac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:47:21 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56780
I20260809 22:47:21.318965 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 10758
I20260809 22:47:21.324505 11559 tablet_replica.cc:330] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: stopping tablet replica
I20260809 22:47:21.325111 11559 raft_consensus.cc:2229] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:47:21.325812 11559 raft_consensus.cc:2258] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:47:21.326845 11562 tablet_replica.cc:330] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: stopping tablet replica
I20260809 22:47:21.327878 11558 tablet_replica.cc:330] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: stopping tablet replica
I20260809 22:47:21.328215 11558 raft_consensus.cc:2229] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:47:21.328259 11559 ts_tablet_manager.cc:1905] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:47:21.328662 11558 raft_consensus.cc:2258] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:47:21.327358 11562 raft_consensus.cc:2229] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:47:21.329685 11562 raft_consensus.cc:2258] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:47:21.331179 11558 ts_tablet_manager.cc:1905] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:47:21.335837 11562 ts_tablet_manager.cc:1905] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:47:21.338884 11558 ts_tablet_manager.cc:1918] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 09 22:47:21 dist-test-slave-pww0 krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:47:21.339249 11558 log.cc:1196] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/f53a950fdbe946a1a1aff3029d3abbac
I20260809 22:47:21.339965 11558 ts_tablet_manager.cc:1939] T f53a950fdbe946a1a1aff3029d3abbac P f683e6fc72a84154aa4843956331cee0: Deleting consensus metadata
I20260809 22:47:21.342473 10626 catalog_manager.cc:4841] TS f683e6fc72a84154aa4843956331cee0 (127.24.136.3:40999): tablet f53a950fdbe946a1a1aff3029d3abbac (table test-table [id=3715b9bee64d4827867bbc58f320a24c]) successfully deleted
I20260809 22:47:21.343493 11559 ts_tablet_manager.cc:1918] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:47:21.343719 11559 log.cc:1196] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/f53a950fdbe946a1a1aff3029d3abbac
I20260809 22:47:21.344338 11559 ts_tablet_manager.cc:1939] T f53a950fdbe946a1a1aff3029d3abbac P a9395a9ebb3e448190225f1484bfb835: Deleting consensus metadata
I20260809 22:47:21.346483 10627 catalog_manager.cc:4841] TS a9395a9ebb3e448190225f1484bfb835 (127.24.136.2:40435): tablet f53a950fdbe946a1a1aff3029d3abbac (table test-table [id=3715b9bee64d4827867bbc58f320a24c]) successfully deleted
I20260809 22:47:21.347976 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.320276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56794 (local address 127.24.136.1:39429)
0809 22:47:21.321035 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:21.321041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:21.324101 (+ 3060us) server_negotiation.cc:407] Connection header received
0809 22:47:21.324373 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:21.324381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:21.324481 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:21.324561 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:21.330176 (+ 5615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.331432 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.333831 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.334578 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.334697 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:21.340321 (+ 5624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:21.340340 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:21.340362 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:21.340420 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:21.344456 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:21.345288 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:21.345297 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:21.345299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:21.345471 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:21.345913 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:21.345917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:21.345919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:21.346409 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:21.346733 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:21.347288 (+ 555us) server_negotiation.cc:299] Negotiation successful
0809 22:47:21.347594 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588}
I20260809 22:47:21.348742 10964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:21.449836 11562 ts_tablet_manager.cc:1918] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:47:21.450181 11562 log.cc:1196] T f53a950fdbe946a1a1aff3029d3abbac P 01f9685bbe09441f99e35aa43c863406: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/f53a950fdbe946a1a1aff3029d3abbac
I20260809 22:47:21.450515 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 10758
W20260809 22:47:21.477382 10627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.136.1:39429 (error 108)
I20260809 22:47:21.477470 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 10980
I20260809 22:47:21.479327 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.478638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39429 (local address 127.0.0.1:56798)
0809 22:47:21.479058 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.479165 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":209,"threads_started":1}
W20260809 22:47:21.479660 10627 catalog_manager.cc:4584] TS 01f9685bbe09441f99e35aa43c863406 (127.24.136.1:39429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f53a950fdbe946a1a1aff3029d3abbac: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39429: connect: Connection refused (error 111)
Aug 09 22:47:21 dist-test-slave-pww0 krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:47:21.493458 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.478204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.24.136.2:40435)
0809 22:47:21.478423 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:21.478430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:21.478527 (+ 97us) server_negotiation.cc:407] Connection header received
0809 22:47:21.478734 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:21.478738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:21.478810 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:21.478910 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:21.480475 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.481237 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.482516 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.482928 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.483045 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:21.487430 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:21.487448 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:21.487552 (+ 104us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:21.487603 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:21.490438 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:21.491208 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:21.491216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:21.491218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:21.491366 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:21.491797 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:21.491801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:21.491803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:21.492227 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:21.492530 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:21.492878 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 22:47:21.493112 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:47:21.494230 11219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:21.533489 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.533005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39429 (local address 127.0.0.1:56802)
0809 22:47:21.533241 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.533338 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 22:47:21.534785 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.534349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39429 (local address 127.0.0.1:56810)
0809 22:47:21.534526 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.534619 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 22:47:21.585479 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 10980
I20260809 22:47:21.595378 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.594815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39429 (local address 127.0.0.1:56822)
0809 22:47:21.595076 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.595189 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 22:47:21.596578 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.596121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39429 (local address 127.0.0.1:56836)
0809 22:47:21.596293 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.596380 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 22:47:21.611047 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 11238
Aug 09 22:47:21 dist-test-slave-pww0 krb5kdc[10579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:47:21.627874 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.611888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.24.136.3:40999)
0809 22:47:21.612069 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:21.612076 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:21.612147 (+ 71us) server_negotiation.cc:407] Connection header received
0809 22:47:21.612352 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:21.612356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:21.612438 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:21.612527 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:21.613940 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.614685 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.616308 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:21.616714 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:21.616837 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:21.621888 (+ 5051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:21.621907 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:21.621925 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:21.621971 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:21.624895 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:21.625668 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:21.625676 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:21.625679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:21.625842 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:21.626248 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:21.626253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:21.626255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:21.626642 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:21.626932 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:21.627270 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 22:47:21.627530 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:47:21.628614 11472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:21.677146 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.676550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39429 (local address 127.0.0.1:56852)
0809 22:47:21.676860 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.676969 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260809 22:47:21.678519 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.677934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39429 (local address 127.0.0.1:56854)
0809 22:47:21.678211 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.678314 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260809 22:47:21.720490 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 11238
I20260809 22:47:21.747251 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 10599
I20260809 22:47:21.748672 10711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:21.863546 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:21.863066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39429 (local address 127.0.0.1:44850)
0809 22:47:21.863242 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:21.863344 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:39429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 22:47:21.863595 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 10599
2026-08-09T22:47:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4871 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:47:21 dist-test-slave-pww0 krb5kdc[11578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:47:21 dist-test-slave-pww0 krb5kdc[11578](info): set up 2 sockets
Aug 09 22:47:21 dist-test-slave-pww0 krb5kdc[11578](info): commencing operation
krb5kdc: starting...
W20260809 22:47:23.141836 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:47:23 dist-test-slave-pww0 krb5kdc[11578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:47:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:47:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.136.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.136.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/rangeradmin_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/rangeradmin_127.24.136.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.136.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.136.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/rangerlookup_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/rangerlookup_127.24.136.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.136.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.136.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/HTTP_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/HTTP_127.24.136.20@KRBTEST.COM.keytab.
I20260809 22:47:23.566694 25120 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknxBkGy/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/postgres -l logfile start
2026-08-09 22:47:26.216 UTC [11638] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 22:47:26.217 UTC [11638] LOG: listening on IPv4 address "127.24.136.20", port 47891
2026-08-09 22:47:26.220 UTC [11638] LOG: listening on Unix socket "/tmp/.s.PGSQL.47891"
2026-08-09 22:47:26.245 UTC [11649] LOG: database system was shut down at 2026-08-09 22:47:24 UTC
2026-08-09 22:47:26.249 UTC [11638] LOG: database system is ready to accept connections
I20260809 22:47:27.391186 25120 mini_postgres.cc:96] Postgres bound to 47891
2026-08-09 22:47:27.417 UTC [11667] FATAL: database "slave" does not exist
127.24.136.20:47891 - accepting connections
I20260809 22:47:27.418253 25120 mini_ranger.cc:161] Starting Ranger...
I20260809 22:47:27.458681 25120 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 22:47:27.961491 25120 mini_ranger.cc:87] Created ranger Postgres database
I20260809 22:47:27.961665 25120 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-admin
2026-08-09 22:47:28,303 [I] DB FLAVOR :POSTGRES
2026-08-09 22:47:28,304 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:47:28,304 [I] Checking connection..
2026-08-09 22:47:28,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 22:47:28,582 [I] Checking connection passed.
2026-08-09 22:47:28,583 [I] --------- Verifying version history table ---------
2026-08-09 22:47:28,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:47:28,866 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:47:28,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:47:29,132 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:47:29,132 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 22:47:29,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 22:47:29.399 UTC [11793] WARNING: there is no transaction in progress
2026-08-09 22:47:29,408 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 22:47:29,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:47:29,690 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 22:47:29,691 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:47:29,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 22:47:29,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 22:47:30,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pww0.c.gcp-upstream.internal','N') ;"
2026-08-09 22:47:30,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:47:30,748 [I] Table x_portal_user does not exist in database ranger
2026-08-09 22:47:30,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:47:31,040 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 22:47:31,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:47:31,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:47:31,591 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 22:47:31,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 22:47:32.620 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.636 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.654 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.671 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.689 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.707 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.765 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.772 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.787 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.797 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.808 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.822 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.832 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.839 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.846 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:32.853 UTC [11973] WARNING: there is no transaction in progress
2026-08-09 22:47:33,547 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 22:47:33,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 22:47:33,791 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 22:47:33,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:47:34,050 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 22:47:34,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:47:34,317 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 22:47:34,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:47:34,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:47:34,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pww0.c.gcp-upstream.internal';"
2026-08-09 22:47:35,129 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 22:47:35,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 22:47:35,388 [I] DB_PATCHES have already been applied
I20260809 22:47:35.395085 25120 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknxBkGy/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 22:47:35.395200 25120 mini_ranger.cc:193] Using host: 127.24.136.20
I20260809 22:47:35.416708 25120 mini_ranger.cc:227] Ranger admin URL: http://127.24.136.20:58755
Aug 09, 2026 10:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 10:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 10:47:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 10:47:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 10:47:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 22:47:36.697827 25120 mini_ranger.cc:160] Time spent starting Ranger: real 9.280s user 0.007s sys 0.069s
Aug 09, 2026 10:47:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 10:47:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 10:47:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 10:47:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 10:47:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 22:47:59.947186 25120 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.136.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.136.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/rangerkms_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/rangerkms_127.24.136.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.136.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.136.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.136.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/HTTP_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.136.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/HTTP_127.24.136.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 22:48:00 dist-test-slave-pww0 krb5kdc[11578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315680, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 22:48:00.180058 25120 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 22:48:00.209228 25120 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 22:48:01.143174 25120 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 22:48:01.143368 25120 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-kms
I20260809 22:48:01.143541 25120 mini_ranger_kms.cc:227] Using postgres at 127.24.136.20:47891
2026-08-09 22:48:01,660 [I] DB FLAVOR :POSTGRES
2026-08-09 22:48:01,661 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:48:01,661 [I] Checking connection
2026-08-09 22:48:01,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 22:48:01,932 [I] connection success
2026-08-09 22:48:01,932 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 22:48:01,932 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 22:48:01,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 22:48:02,214 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 22:48:02,215 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:48:02,215 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 22:48:02,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:47891/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 22:48:02,529 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 22:48:03.021492 25120 mini_ranger_kms.cc:310] Created kms service
I20260809 22:48:03.277268 25120 mini_ranger_kms.cc:326] Created kudu user
I20260809 22:48:03.326174 25120 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 22:48:03 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315680, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:48:03.726308 25120 mini_ranger_kms.cc:384] Added ranger policy
I20260809 22:48:03.726698 25120 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 22:48:03.726773 25120 mini_ranger_kms.cc:240] Using host: 127.24.136.20
I20260809 22:48:03.753356 25120 mini_ranger_kms.cc:276] Ranger KMS PID: 12297
I20260809 22:48:03.753647 25120 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.136.20:56309
Aug 09, 2026 10:48:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:48:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:48:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 10:48:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 10:48:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 10:48:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 22:48:06 dist-test-slave-pww0 krb5kdc[11578](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315686, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.136.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:06.406105 25120 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.226s user 0.087s sys 0.477s
I20260809 22:48:06.406507 25120 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 22:48:06.406737 25120 mini_ranger_kms.cc:401] 127.24.136.20:56309/kms/v1/keys
Aug 09 22:48:06 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315686, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.136.20@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
Aug 09 22:48:06 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315686, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.136.20@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
Aug 09 22:48:09 dist-test-slave-pww0 krb5kdc[11578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:48:09.699242 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:33823
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:33823
--ranger_config_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:56309/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:10.072939 12386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:10.073251 12386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:10.073366 12386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:10.081501 12386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:10.081638 12386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:10.081697 12386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:10.081777 12386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:10.081843 12386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:10.092929 12386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.136.20:56309/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:33823
--ranger_config_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:33823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:48:10.094386 12386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:10.095853 12386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:10.110977 12399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:10.110960 12398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:10.111306 12402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:10.111575 12386 server_base.cc:1034] running on GCE node
I20260809 22:48:10.112469 12386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:10.114077 12386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:10.115322 12386 hybrid_clock.cc:648] HybridClock initialized: now 1786315690115277 us; error 42 us; skew 500 ppm
I20260809 22:48:10.115751 12386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:10 dist-test-slave-pww0 krb5kdc[11578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315690, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:10.119176 12386 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:48:10.120934 12386 webserver.cc:460] Webserver started at http://127.24.136.62:37967/ using document root <none> and password file <none>
I20260809 22:48:10.121644 12386 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:10.121765 12386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:10.122076 12386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:48:10 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315690, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:48:10.177073 12386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "5dbe4d5dfaae4904abd7080011c3aee3"
format_stamp: "Formatted at 2026-08-09 22:48:10 on dist-test-slave-pww0"
server_key: "a20673f15a003f7d055b324af223ad82"
server_key_iv: "28c4ff5ee6ee0e33dbba92d03b0968f3"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:10.178082 12386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "5dbe4d5dfaae4904abd7080011c3aee3"
format_stamp: "Formatted at 2026-08-09 22:48:10 on dist-test-slave-pww0"
server_key: "a20673f15a003f7d055b324af223ad82"
server_key_iv: "28c4ff5ee6ee0e33dbba92d03b0968f3"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:10.184166 12386 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260809 22:48:10.201473 12411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:10.203032 12386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:48:10.203222 12386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "5dbe4d5dfaae4904abd7080011c3aee3"
format_stamp: "Formatted at 2026-08-09 22:48:10 on dist-test-slave-pww0"
server_key: "a20673f15a003f7d055b324af223ad82"
server_key_iv: "28c4ff5ee6ee0e33dbba92d03b0968f3"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:10.203487 12386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:10.236889 12386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:10.241616 12386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:10.241931 12386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:10.262219 12386 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:33823
I20260809 22:48:10.262238 12489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:33823 every 8 connection(s)
I20260809 22:48:10.263747 12386 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:10.266552 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 12386
I20260809 22:48:10.266845 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:48:10.267777 25120 external_mini_cluster.cc:1428] Setting key 882c59db702a15572f711860d80987a8
I20260809 22:48:10.268190 12491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:10.285475 12491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3: Bootstrap starting.
I20260809 22:48:10.289583 12491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:10.291108 12491 log.cc:824] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:10.294795 12491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3: No bootstrap required, opened a new log
Aug 09 22:48:10 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:48:10.304615 12491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dbe4d5dfaae4904abd7080011c3aee3" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33823 } }
I20260809 22:48:10.305181 12491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:10.305315 12491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dbe4d5dfaae4904abd7080011c3aee3, State: Initialized, Role: FOLLOWER
I20260809 22:48:10.306228 12491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dbe4d5dfaae4904abd7080011c3aee3" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33823 } }
I20260809 22:48:10.306555 12491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:10.306751 12491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:10.306941 12491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:10.309877 12491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dbe4d5dfaae4904abd7080011c3aee3" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33823 } }
I20260809 22:48:10.310541 12491 leader_election.cc:302] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dbe4d5dfaae4904abd7080011c3aee3; no voters:
I20260809 22:48:10.311226 12491 leader_election.cc:288] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:10.311406 12500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:10.311697 12497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:10.270160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.24.136.62:33823)
0809 22:48:10.281941 (+ 11781us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:10.281964 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:10.282913 (+ 949us) server_negotiation.cc:407] Connection header received
0809 22:48:10.285540 (+ 2627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:10.285575 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:10.286189 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:10.286691 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:10.288480 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:10.290595 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:10.292640 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:10.293708 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:10.293929 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:10.299192 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:10.299205 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:10.299232 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:10.299292 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:10.304258 (+ 4966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:10.305525 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:10.305536 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:10.305546 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:10.305786 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:10.306421 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:10.306432 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:10.306436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:10.307503 (+ 1067us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:10.307887 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:10.309495 (+ 1608us) server_negotiation.cc:299] Negotiation successful
0809 22:48:10.310018 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11439,"thread_start_us":11706,"threads_started":1}
I20260809 22:48:10.316215 12500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [term 1 LEADER]: Becoming Leader. State: Replica: 5dbe4d5dfaae4904abd7080011c3aee3, State: Running, Role: LEADER
I20260809 22:48:10.317260 12500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dbe4d5dfaae4904abd7080011c3aee3" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33823 } }
I20260809 22:48:10.326701 12491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:10.328286 12506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dbe4d5dfaae4904abd7080011c3aee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dbe4d5dfaae4904abd7080011c3aee3" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33823 } } }
I20260809 22:48:10.328936 12506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:10.329591 12501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dbe4d5dfaae4904abd7080011c3aee3. Latest consensus state: current_term: 1 leader_uuid: "5dbe4d5dfaae4904abd7080011c3aee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dbe4d5dfaae4904abd7080011c3aee3" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33823 } } }
I20260809 22:48:10.329862 12501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:10.333774 12511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:10.339951 12511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:10.340983 12491 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 22:48:10.359613 12511 catalog_manager.cc:1347] Generated new cluster ID: bf0cb0af2ac348ab8f088871d9a26cd7
I20260809 22:48:10.359793 12511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:10.384706 12511 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:10.386965 12511 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:10.399050 12511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3: Generated new TSK 0
I20260809 22:48:10.400313 12511 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:48:11.165409 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:56309/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33823
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:48:11 dist-test-slave-pww0 krb5kdc[11578](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315691, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 22:48:11.631718 12569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:11.632187 12569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:11.632413 12569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:11.644704 12569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:11.644886 12569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:11.645139 12569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:48:11.662145 12569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.136.20:56309/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:48:11.664191 12569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:11.666002 12569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:11.689703 12592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:11.695259 12593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:11.696884 12596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:11.697160 12569 server_base.cc:1034] running on GCE node
I20260809 22:48:11.697854 12569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:11.701032 12569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:11.702279 12569 hybrid_clock.cc:648] HybridClock initialized: now 1786315691702203 us; error 73 us; skew 500 ppm
I20260809 22:48:11.702696 12569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:11 dist-test-slave-pww0 krb5kdc[11578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315691, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:11.706902 12569 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:48:11.708799 12569 webserver.cc:460] Webserver started at http://127.24.136.1:45185/ using document root <none> and password file <none>
I20260809 22:48:11.709546 12569 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:11.709671 12569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:11.709981 12569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:48:11 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315691, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:48:11.730211 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "8c9f7e54191846bc9ba4e5c8af2aba90"
format_stamp: "Formatted at 2026-08-09 22:48:11 on dist-test-slave-pww0"
server_key: "826b12262de32f363e6b694775dbd49a"
server_key_iv: "95ec19d616ebef8d60c4668f81d67574"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:11.731842 12569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "8c9f7e54191846bc9ba4e5c8af2aba90"
format_stamp: "Formatted at 2026-08-09 22:48:11 on dist-test-slave-pww0"
server_key: "826b12262de32f363e6b694775dbd49a"
server_key_iv: "95ec19d616ebef8d60c4668f81d67574"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:11.747223 12569 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.011s sys 0.000s
I20260809 22:48:11.781546 12605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:11.783771 12569 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.010s sys 0.000s
I20260809 22:48:11.784032 12569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "8c9f7e54191846bc9ba4e5c8af2aba90"
format_stamp: "Formatted at 2026-08-09 22:48:11 on dist-test-slave-pww0"
server_key: "826b12262de32f363e6b694775dbd49a"
server_key_iv: "95ec19d616ebef8d60c4668f81d67574"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:11.784345 12569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:11.838992 12569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:11.844946 12569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:11.845396 12569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:11.859979 12569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:11.868489 12569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:11.868700 12569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:48:11.868907 12569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:11.869022 12569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:48:11 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315691, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
Aug 09 22:48:12 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315691, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:48:12.531333 12569 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:45301
I20260809 22:48:12.531395 12802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:45301 every 8 connection(s)
I20260809 22:48:12.533179 12569 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:48:12.534273 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 12569
I20260809 22:48:12.534500 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:48:12.535601 25120 external_mini_cluster.cc:1428] Setting key a841380c07c9051c1441436d5ff1feb0
Aug 09 22:48:12 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315691, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
I20260809 22:48:12.601874 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:12.561731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:54377 (local address 127.24.136.62:33823)
0809 22:48:12.576065 (+ 14334us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:12.576073 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:12.576100 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:48:12.576174 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:12.576179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:12.576276 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:12.578084 (+ 1808us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:12.583390 (+ 5306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:12.584548 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:12.587585 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:12.588317 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:12.588455 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:12.593639 (+ 5184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:12.593649 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:12.593653 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:12.593718 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:12.597381 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:12.598585 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:12.598591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:12.598594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:12.598785 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:12.599426 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:12.599432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:12.599435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:12.599839 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:12.599977 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:12.601163 (+ 1186us) server_negotiation.cc:299] Negotiation successful
0809 22:48:12.601451 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14156,"thread_start_us":21305,"threads_started":1}
I20260809 22:48:12.603552 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:12.562347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33823 (local address 127.24.136.1:54377)
0809 22:48:12.572907 (+ 10560us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:12.572969 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:12.575441 (+ 2472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:12.578436 (+ 2995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:12.578451 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:12.579234 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:12.580296 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:12.580329 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:12.584883 (+ 4554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:12.584889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:12.587244 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:12.587274 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:12.588864 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:12.588872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:12.589206 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:12.590154 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:12.590191 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:12.593264 (+ 3073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:12.597648 (+ 4384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:12.597659 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:12.597692 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:12.598400 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:12.598943 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:12.598949 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:12.598953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:12.599270 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:12.600115 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:12.600125 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:12.600625 (+ 500us) client_negotiation.cc:769] Sending connection context
0809 22:48:12.600974 (+ 349us) client_negotiation.cc:240] Negotiation successful
0809 22:48:12.601571 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10203,"thread_start_us":9962,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
I20260809 22:48:12.605760 12804 heartbeater.cc:347] Connected to a master server at 127.24.136.62:33823
I20260809 22:48:12.606164 12804 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:12.607277 12804 heartbeater.cc:511] Master 127.24.136.62:33823 requested a full tablet report, sending...
I20260809 22:48:12.610313 12433 ts_manager.cc:194] Registered new tserver with Master: 8c9f7e54191846bc9ba4e5c8af2aba90 (127.24.136.1:45301)
I20260809 22:48:12.612488 12433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:54377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
W20260809 22:48:12.652570 25120 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.136.2: real 0.113s user 0.000s sys 0.027s
I20260809 22:48:12.653661 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:56309/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33823
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:13.055265 12818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:13.055651 12818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:13.055811 12818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:13.063449 12818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:13.063642 12818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:13.063798 12818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:48:13.074767 12818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.136.20:56309/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:48:13.076408 12818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:13.077836 12818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:13.094137 12841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:13.094211 12842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:13.094157 12845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:13.094873 12818 server_base.cc:1034] running on GCE node
I20260809 22:48:13.095646 12818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:13.096828 12818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:13.098089 12818 hybrid_clock.cc:648] HybridClock initialized: now 1786315693098002 us; error 70 us; skew 500 ppm
I20260809 22:48:13.098536 12818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:13 dist-test-slave-pww0 krb5kdc[11578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:13.106758 12818 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:48:13.108819 12818 webserver.cc:460] Webserver started at http://127.24.136.2:45643/ using document root <none> and password file <none>
I20260809 22:48:13.109818 12818 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:13.109972 12818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:13.110368 12818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:48:13 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:48:13.130599 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "b03353283bc1450c914bf953c840e3a5"
format_stamp: "Formatted at 2026-08-09 22:48:13 on dist-test-slave-pww0"
server_key: "966ffcd894ccf7a5b6fc7c37831d80fc"
server_key_iv: "21a5885939687e78ffeb83bf4e8b216d"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:13.131971 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "b03353283bc1450c914bf953c840e3a5"
format_stamp: "Formatted at 2026-08-09 22:48:13 on dist-test-slave-pww0"
server_key: "966ffcd894ccf7a5b6fc7c37831d80fc"
server_key_iv: "21a5885939687e78ffeb83bf4e8b216d"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:13.139894 12818 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260809 22:48:13.159966 12855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:13.161953 12818 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:48:13.162159 12818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "b03353283bc1450c914bf953c840e3a5"
format_stamp: "Formatted at 2026-08-09 22:48:13 on dist-test-slave-pww0"
server_key: "966ffcd894ccf7a5b6fc7c37831d80fc"
server_key_iv: "21a5885939687e78ffeb83bf4e8b216d"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:13.162433 12818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:13.186058 12818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:13.191941 12818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:13.192337 12818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:13.193698 12818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:13.195755 12818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:13.195861 12818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:13.196049 12818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:13.196132 12818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:13.240409 12818 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:43663
I20260809 22:48:13.240412 13040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:43663 every 8 connection(s)
I20260809 22:48:13.241796 12818 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:48:13.244254 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 12818
I20260809 22:48:13.244419 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:48:13.244864 25120 external_mini_cluster.cc:1428] Setting key bc45d6f2bee6dd8f9cd6561da937aad6
Aug 09 22:48:13 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:48:13.274955 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.245917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:33695 (local address 127.24.136.62:33823)
0809 22:48:13.255929 (+ 10012us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:13.255933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:13.255956 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:48:13.256019 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:13.256022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:13.256104 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:13.256222 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:13.258613 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.259528 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.261266 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.261920 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.262045 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.266879 (+ 4834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:13.266886 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:13.266890 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:13.266934 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:13.270722 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:13.271858 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:13.271863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:13.271866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:13.272027 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:13.272570 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:13.272587 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:13.272591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:13.273052 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:13.273182 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:13.274291 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0809 22:48:13.274552 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9882,"thread_start_us":9775,"threads_started":1}
I20260809 22:48:13.276369 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.245337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33823 (local address 127.24.136.2:33695)
0809 22:48:13.246268 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:13.246322 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:13.248622 (+ 2300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:13.256683 (+ 8061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:13.256698 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:13.257402 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:13.258353 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.258387 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.259733 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.259737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.261085 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.261100 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.262371 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.262378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.262729 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.263587 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:13.263621 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:13.266549 (+ 2928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:13.270961 (+ 4412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:13.270971 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:13.270991 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:13.271693 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:13.272162 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:13.272167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:13.272170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:13.272426 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:13.273277 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:13.273285 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:13.273789 (+ 504us) client_negotiation.cc:769] Sending connection context
0809 22:48:13.274116 (+ 327us) client_negotiation.cc:240] Negotiation successful
0809 22:48:13.274797 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":580,"threads_started":1}
I20260809 22:48:13.278182 13041 heartbeater.cc:347] Connected to a master server at 127.24.136.62:33823
I20260809 22:48:13.278605 13041 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:13.279708 13041 heartbeater.cc:511] Master 127.24.136.62:33823 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
I20260809 22:48:13.281870 12433 ts_manager.cc:194] Registered new tserver with Master: b03353283bc1450c914bf953c840e3a5 (127.24.136.2:43663)
I20260809 22:48:13.282739 12433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:33695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:48:13.320858 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:56309/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33823
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:46047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:48:13.617043 12804 heartbeater.cc:503] Master 127.24.136.62:33823 was elected leader, sending a full tablet report...
W20260809 22:48:13.682981 13053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:13.683305 13053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:13.683405 13053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:13.691095 13053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:13.691232 13053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:13.691385 13053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:48:13.702016 13053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:46047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.136.20:56309/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:48:13.703438 13053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:13.704864 13053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:13.718907 13064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:13.718797 13067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:13.718833 13065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:13.719830 13053 server_base.cc:1034] running on GCE node
I20260809 22:48:13.720503 13053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:13.721592 13053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:13.722813 13053 hybrid_clock.cc:648] HybridClock initialized: now 1786315693722760 us; error 53 us; skew 500 ppm
I20260809 22:48:13.723161 13053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:13 dist-test-slave-pww0 krb5kdc[11578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:13.726864 13053 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:48:13.728669 13053 webserver.cc:460] Webserver started at http://127.24.136.3:41233/ using document root <none> and password file <none>
I20260809 22:48:13.729473 13053 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:13.729624 13053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:13.729941 13053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:48:13 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:48:13.746075 13053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "58729dc203d7471ca95c0da16b8edf76"
format_stamp: "Formatted at 2026-08-09 22:48:13 on dist-test-slave-pww0"
server_key: "2011aefb12fd09d71ed1758a8dc76542"
server_key_iv: "1fb5cffb22f42267a11db6d259789094"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:13.747186 13053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "58729dc203d7471ca95c0da16b8edf76"
format_stamp: "Formatted at 2026-08-09 22:48:13 on dist-test-slave-pww0"
server_key: "2011aefb12fd09d71ed1758a8dc76542"
server_key_iv: "1fb5cffb22f42267a11db6d259789094"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:13.753728 13053 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 22:48:13.773082 13076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:13.775004 13053 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:48:13.775213 13053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "58729dc203d7471ca95c0da16b8edf76"
format_stamp: "Formatted at 2026-08-09 22:48:13 on dist-test-slave-pww0"
server_key: "2011aefb12fd09d71ed1758a8dc76542"
server_key_iv: "1fb5cffb22f42267a11db6d259789094"
server_key_version: "kuduclusterkey@0"
I20260809 22:48:13.775542 13053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:13.792737 13053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:13.796983 13053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:13.797281 13053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:13.798261 13053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:13.799788 13053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:13.799865 13053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:13.799952 13053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:13.799999 13053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:13.834112 13053 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:33887
I20260809 22:48:13.834210 13265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:33887 every 8 connection(s)
I20260809 22:48:13.835567 13053 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:48:13.841492 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 13053
I20260809 22:48:13.841689 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:48:13.842247 25120 external_mini_cluster.cc:1428] Setting key 0a3b84d138d723fd34fb5fa0a7ed4f68
Aug 09 22:48:13 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315693, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:48:13.857461 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.838268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:47241 (local address 127.24.136.62:33823)
0809 22:48:13.838826 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:13.838833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:13.839922 (+ 1089us) server_negotiation.cc:407] Connection header received
0809 22:48:13.842101 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:13.842107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:13.842206 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:13.842358 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:13.844178 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.845378 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.847167 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.847690 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.847794 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.851278 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:13.851284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:13.851287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:13.851326 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:13.854081 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:13.854944 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:13.854948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:13.854951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:13.855097 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:13.855556 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:13.855559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:13.855561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:13.855866 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:13.855983 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:13.856851 (+ 868us) server_negotiation.cc:299] Negotiation successful
0809 22:48:13.857111 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":235,"threads_started":1}
I20260809 22:48:13.858608 13272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.838890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33823 (local address 127.24.136.3:47241)
0809 22:48:13.839747 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:13.839788 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:13.841711 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:13.842640 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:13.842653 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:13.843183 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:13.843952 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.843983 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.845574 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.845579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.846987 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.847009 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.847940 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.847947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.848173 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.848842 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:13.848867 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:13.851005 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:13.854267 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:13.854276 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:13.854294 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:13.854810 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:13.855201 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:13.855206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:13.855209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:13.855419 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:13.855970 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:13.855978 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:13.856449 (+ 471us) client_negotiation.cc:769] Sending connection context
0809 22:48:13.856732 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 22:48:13.857279 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":330,"threads_started":1}
I20260809 22:48:13.859983 13267 heartbeater.cc:347] Connected to a master server at 127.24.136.62:33823
I20260809 22:48:13.860306 13267 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:13.861080 13267 heartbeater.cc:511] Master 127.24.136.62:33823 requested a full tablet report, sending...
I20260809 22:48:13.862637 12433 ts_manager.cc:194] Registered new tserver with Master: 58729dc203d7471ca95c0da16b8edf76 (127.24.136.3:33887)
I20260809 22:48:13.863498 12433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:47241
I20260809 22:48:13.871551 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:48:13.889734 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.876584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.24.136.62:33823)
0809 22:48:13.876768 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:13.876773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:13.877271 (+ 498us) server_negotiation.cc:407] Connection header received
0809 22:48:13.877409 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:13.877413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:13.877501 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:13.877623 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:13.879000 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.879860 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.881391 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.882027 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.882140 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.884571 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:13.884596 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:13.884600 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:13.884641 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:13.887375 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:13.888079 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:13.888084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:13.888088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:13.888231 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:13.888545 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:13.888548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:13.888550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:13.888837 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:13.889021 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:13.889279 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 22:48:13.889476 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:48:13.894152 12433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:48:13.901512 12433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:48:13.927662 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.915512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.24.136.1:45301)
0809 22:48:13.918652 (+ 3140us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:13.918661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:13.918690 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:48:13.918801 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:13.918809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:13.919100 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:13.919964 (+ 864us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:13.920171 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.921247 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.924894 (+ 3647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.926374 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.926495 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.926746 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:13.927080 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 22:48:13.927250 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2960,"thread_start_us":2571,"threads_started":1}
I20260809 22:48:13.927752 13290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.914963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45301 (local address 127.0.0.1:56052)
0809 22:48:13.916816 (+ 1853us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:13.916868 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:13.917061 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:13.919353 (+ 2292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:13.919362 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:13.919397 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:13.919778 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.919787 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.921392 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.921397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.924718 (+ 3321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.924733 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.926595 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.926602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.926831 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.926872 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:48:13.926996 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 22:48:13.927222 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1579,"mutex_wait_us":1,"thread_start_us":2397,"threads_started":1}
I20260809 22:48:13.930423 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.915124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:33887 (local address 127.0.0.1:40782)
0809 22:48:13.917622 (+ 2498us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:13.917641 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:13.917757 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:13.922959 (+ 5202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:13.922965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:13.922991 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:13.923905 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.923915 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.925566 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.925572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.927682 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.927695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.929627 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.929634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.929811 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.929845 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:48:13.929933 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:48:13.930063 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2379,"thread_start_us":1265,"threads_started":1}
I20260809 22:48:13.930578 13294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.915941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40782 (local address 127.24.136.3:33887)
0809 22:48:13.922304 (+ 6363us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:13.922314 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:13.922348 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:48:13.922444 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:13.922452 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:13.922733 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:13.923097 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:13.924117 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.925428 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.927891 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.929361 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.929494 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.929840 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:13.930021 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 22:48:13.930188 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6187,"thread_start_us":6028,"threads_started":1}
I20260809 22:48:13.932272 13292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.915696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:43663 (local address 127.0.0.1:45134)
0809 22:48:13.922626 (+ 6930us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:13.922644 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:13.922783 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:13.923506 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:13.923511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:13.923537 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:13.924363 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.924373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.926108 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.926114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.929512 (+ 3398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:13.929532 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.931451 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.931483 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:13.931649 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.931680 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:48:13.931760 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 22:48:13.931886 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6814,"mutex_wait_us":26,"thread_start_us":6670,"threads_started":1}
I20260809 22:48:13.932981 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.916100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.24.136.2:43663)
0809 22:48:13.921799 (+ 5699us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:13.921809 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:13.922939 (+ 1130us) server_negotiation.cc:407] Connection header received
0809 22:48:13.923057 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:13.923065 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:13.923294 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:13.923523 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:13.924532 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.925970 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.929714 (+ 3744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:13.931220 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:13.931360 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:13.931571 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:13.932537 (+ 966us) server_negotiation.cc:299] Negotiation successful
0809 22:48:13.932708 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5523,"thread_start_us":5416,"threads_started":1}
I20260809 22:48:13.932551 12688 tablet_service.cc:1467] Processing CreateTablet for tablet 705aa0b6128d4b718c491610333feaca (DEFAULT_TABLE table=test-table [id=14bacbabaab948f0bec89c2d2592e9a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:13.934072 12688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705aa0b6128d4b718c491610333feaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:13.934356 13164 tablet_service.cc:1467] Processing CreateTablet for tablet 705aa0b6128d4b718c491610333feaca (DEFAULT_TABLE table=test-table [id=14bacbabaab948f0bec89c2d2592e9a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:13.935632 13164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705aa0b6128d4b718c491610333feaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:13.939842 12930 tablet_service.cc:1467] Processing CreateTablet for tablet 705aa0b6128d4b718c491610333feaca (DEFAULT_TABLE table=test-table [id=14bacbabaab948f0bec89c2d2592e9a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:13.941399 12930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 705aa0b6128d4b718c491610333feaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:13.948362 13301 tablet_bootstrap.cc:492] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: Bootstrap starting.
I20260809 22:48:13.951891 13301 tablet_bootstrap.cc:654] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:13.952310 13303 tablet_bootstrap.cc:492] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: Bootstrap starting.
I20260809 22:48:13.953538 13301 log.cc:824] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:13.956328 13303 tablet_bootstrap.cc:654] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:13.957563 13301 tablet_bootstrap.cc:492] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: No bootstrap required, opened a new log
I20260809 22:48:13.957921 13301 ts_tablet_manager.cc:1397] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 22:48:13.957904 13303 log.cc:824] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:13.961591 13303 tablet_bootstrap.cc:492] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: No bootstrap required, opened a new log
I20260809 22:48:13.959558 13310 tablet_bootstrap.cc:492] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: Bootstrap starting.
I20260809 22:48:13.961922 13303 ts_tablet_manager.cc:1397] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260809 22:48:13.964967 13310 tablet_bootstrap.cc:654] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:13.966086 13310 log.cc:824] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:13.968855 13301 raft_consensus.cc:348] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } }
I20260809 22:48:13.969388 13301 raft_consensus.cc:374] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:13.969580 13301 raft_consensus.cc:729] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9f7e54191846bc9ba4e5c8af2aba90, State: Initialized, Role: FOLLOWER
I20260809 22:48:13.970340 13301 consensus_queue.cc:260] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } }
I20260809 22:48:13.971112 13310 tablet_bootstrap.cc:492] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: No bootstrap required, opened a new log
I20260809 22:48:13.971506 13310 ts_tablet_manager.cc:1397] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260809 22:48:13.972388 13301 ts_tablet_manager.cc:1428] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260809 22:48:13.972607 13303 raft_consensus.cc:348] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } }
I20260809 22:48:13.973145 13303 raft_consensus.cc:374] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:13.973251 13303 raft_consensus.cc:729] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58729dc203d7471ca95c0da16b8edf76, State: Initialized, Role: FOLLOWER
I20260809 22:48:13.974125 13303 consensus_queue.cc:260] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } }
I20260809 22:48:13.978152 13267 heartbeater.cc:503] Master 127.24.136.62:33823 was elected leader, sending a full tablet report...
I20260809 22:48:13.978868 13303 ts_tablet_manager.cc:1428] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: Time spent starting tablet: real 0.017s user 0.001s sys 0.014s
I20260809 22:48:13.982699 13310 raft_consensus.cc:348] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } }
I20260809 22:48:13.983103 13310 raft_consensus.cc:374] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:13.983211 13310 raft_consensus.cc:729] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03353283bc1450c914bf953c840e3a5, State: Initialized, Role: FOLLOWER
I20260809 22:48:13.983887 13310 consensus_queue.cc:260] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } }
I20260809 22:48:13.985023 13041 heartbeater.cc:503] Master 127.24.136.62:33823 was elected leader, sending a full tablet report...
I20260809 22:48:13.985539 13310 ts_tablet_manager.cc:1428] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
W20260809 22:48:13.993993 13042 tablet.cc:2369] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:14.059590 13321 raft_consensus.cc:482] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:14.059926 13321 raft_consensus.cc:504] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } }
I20260809 22:48:14.061779 13321 leader_election.cc:288] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c9f7e54191846bc9ba4e5c8af2aba90 (127.24.136.1:45301), b03353283bc1450c914bf953c840e3a5 (127.24.136.2:43663)
I20260809 22:48:14.068982 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.062407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:37039 (local address 127.24.136.2:43663)
0809 22:48:14.062597 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:14.062602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:14.062624 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:48:14.062918 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:14.062924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:14.063026 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:14.063149 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:14.063634 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.064537 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.067022 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.068020 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.068175 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:14.068238 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:14.068575 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 22:48:14.068716 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:48:14.069065 13272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.062187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:43663 (local address 127.24.136.3:37039)
0809 22:48:14.062535 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:14.062563 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:14.062706 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:14.063192 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:14.063198 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:14.063229 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:14.063489 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:14.063504 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.064731 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.064736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:14.066830 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:14.066846 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.068185 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.068191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:14.068403 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:14.068430 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 22:48:14.068523 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:48:14.068687 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":31}
I20260809 22:48:14.069633 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.062316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45301 (local address 127.24.136.3:54185)
0809 22:48:14.063135 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:14.063152 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:14.063334 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:14.063732 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:14.063736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:14.063764 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:14.064050 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:14.064058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.065447 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.065451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:14.066875 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:14.066890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.068384 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.068389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:14.068515 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:14.068538 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:48:14.068686 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 22:48:14.068824 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":2,"spinlock_wait_cycles":26368,"thread_start_us":519,"threads_started":1}
I20260809 22:48:14.069854 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.062470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:54185 (local address 127.24.136.1:45301)
0809 22:48:14.062703 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:14.062710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:14.063257 (+ 547us) server_negotiation.cc:407] Connection header received
0809 22:48:14.063493 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:14.063498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:14.063606 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:14.063711 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:14.064207 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.065264 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.067095 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.068041 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.068194 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:14.068281 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:14.068717 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 22:48:14.068905 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 22:48:14.070333 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "705aa0b6128d4b718c491610333feaca" candidate_uuid: "58729dc203d7471ca95c0da16b8edf76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b03353283bc1450c914bf953c840e3a5" is_pre_election: true
I20260809 22:48:14.070997 12965 raft_consensus.cc:2454] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58729dc203d7471ca95c0da16b8edf76 in term 0.
I20260809 22:48:14.070904 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "705aa0b6128d4b718c491610333feaca" candidate_uuid: "58729dc203d7471ca95c0da16b8edf76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" is_pre_election: true
I20260809 22:48:14.071530 12722 raft_consensus.cc:2454] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58729dc203d7471ca95c0da16b8edf76 in term 0.
I20260809 22:48:14.071870 13082 leader_election.cc:302] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58729dc203d7471ca95c0da16b8edf76, b03353283bc1450c914bf953c840e3a5; no voters:
I20260809 22:48:14.072376 13321 raft_consensus.cc:2781] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:48:14.072537 13321 raft_consensus.cc:482] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:48:14.072647 13321 raft_consensus.cc:3037] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 0 FOLLOWER]: Advancing to term 1
W20260809 22:48:14.073724 12811 tablet.cc:2369] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:14.075312 13321 raft_consensus.cc:504] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } }
I20260809 22:48:14.076002 13321 leader_election.cc:288] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [CANDIDATE]: Term 1 election: Requested vote from peers 8c9f7e54191846bc9ba4e5c8af2aba90 (127.24.136.1:45301), b03353283bc1450c914bf953c840e3a5 (127.24.136.2:43663)
I20260809 22:48:14.076596 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "705aa0b6128d4b718c491610333feaca" candidate_uuid: "58729dc203d7471ca95c0da16b8edf76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b03353283bc1450c914bf953c840e3a5"
I20260809 22:48:14.076596 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "705aa0b6128d4b718c491610333feaca" candidate_uuid: "58729dc203d7471ca95c0da16b8edf76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90"
I20260809 22:48:14.076834 12965 raft_consensus.cc:3037] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:14.076834 12722 raft_consensus.cc:3037] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:14.080211 12722 raft_consensus.cc:2454] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58729dc203d7471ca95c0da16b8edf76 in term 1.
I20260809 22:48:14.080231 12965 raft_consensus.cc:2454] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58729dc203d7471ca95c0da16b8edf76 in term 1.
I20260809 22:48:14.080760 13082 leader_election.cc:302] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 58729dc203d7471ca95c0da16b8edf76, 8c9f7e54191846bc9ba4e5c8af2aba90, b03353283bc1450c914bf953c840e3a5; no voters:
I20260809 22:48:14.081130 13321 raft_consensus.cc:2781] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:14.081645 13321 raft_consensus.cc:686] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 1 LEADER]: Becoming Leader. State: Replica: 58729dc203d7471ca95c0da16b8edf76, State: Running, Role: LEADER
I20260809 22:48:14.082239 13321 consensus_queue.cc:237] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } }
W20260809 22:48:14.087330 13270 tablet.cc:2369] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:14.087222 12433 catalog_manager.cc:5495] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58729dc203d7471ca95c0da16b8edf76 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "58729dc203d7471ca95c0da16b8edf76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58729dc203d7471ca95c0da16b8edf76" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 } health_report { overall_health: UNKNOWN } } }
I20260809 22:48:14.123004 13294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.117530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40784 (local address 127.24.136.3:33887)
0809 22:48:14.117768 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:14.117773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:14.117794 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:48:14.117994 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:14.117998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:14.118082 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:14.118215 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:48:14.118520 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.119314 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.120828 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.121243 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.121348 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:14.121893 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:48:14.122421 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:48:14.122536 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:14.122631 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 22:48:14.122710 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:48:14.132205 12722 raft_consensus.cc:1261] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [term 1 FOLLOWER]: Refusing update from remote peer 58729dc203d7471ca95c0da16b8edf76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:14.132205 12965 raft_consensus.cc:1261] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [term 1 FOLLOWER]: Refusing update from remote peer 58729dc203d7471ca95c0da16b8edf76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:14.133212 13325 consensus_queue.cc:1035] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b03353283bc1450c914bf953c840e3a5" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:14.133602 13321 consensus_queue.cc:1035] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c9f7e54191846bc9ba4e5c8af2aba90" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:14.147137 13329 mvcc.cc:205] Tried to move back new op lower bound from 7316749083154358272 to 7316749082965147648. Current Snapshot: MvccSnapshot[applied={T|T < 7316749083154358272}]
I20260809 22:48:14.161746 12433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:14.162019 12433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:14.165896 12433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dbe4d5dfaae4904abd7080011c3aee3: Sending DeleteTablet for 3 replicas of tablet 705aa0b6128d4b718c491610333feaca
I20260809 22:48:14.167174 12930 tablet_service.cc:1514] Processing DeleteTablet for tablet 705aa0b6128d4b718c491610333feaca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:14 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:45134
I20260809 22:48:14.167172 13164 tablet_service.cc:1514] Processing DeleteTablet for tablet 705aa0b6128d4b718c491610333feaca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:14 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:40782
I20260809 22:48:14.167181 12688 tablet_service.cc:1514] Processing DeleteTablet for tablet 705aa0b6128d4b718c491610333feaca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:14 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56052
I20260809 22:48:14.170143 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 12569
I20260809 22:48:14.178854 13346 tablet_replica.cc:330] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: stopping tablet replica
I20260809 22:48:14.179491 13346 raft_consensus.cc:2229] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:14.180068 13347 tablet_replica.cc:330] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: stopping tablet replica
I20260809 22:48:14.180321 13346 raft_consensus.cc:2258] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:14.180688 13347 raft_consensus.cc:2229] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:14.181262 13345 tablet_replica.cc:330] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: stopping tablet replica
I20260809 22:48:14.181306 13347 raft_consensus.cc:2258] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:14.181720 13345 raft_consensus.cc:2229] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:48:14.182492 13345 raft_consensus.cc:2258] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:14.184079 13347 ts_tablet_manager.cc:1905] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:14.184971 13346 ts_tablet_manager.cc:1905] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:14.192054 13347 ts_tablet_manager.cc:1918] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:14.192387 13347 log.cc:1196] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/705aa0b6128d4b718c491610333feaca
I20260809 22:48:14.192586 13345 ts_tablet_manager.cc:1905] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:14.193076 13346 ts_tablet_manager.cc:1918] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 09 22:48:14 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:48:14.193136 13347 ts_tablet_manager.cc:1939] T 705aa0b6128d4b718c491610333feaca P 8c9f7e54191846bc9ba4e5c8af2aba90: Deleting consensus metadata
I20260809 22:48:14.193284 13346 log.cc:1196] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/705aa0b6128d4b718c491610333feaca
I20260809 22:48:14.193835 13346 ts_tablet_manager.cc:1939] T 705aa0b6128d4b718c491610333feaca P b03353283bc1450c914bf953c840e3a5: Deleting consensus metadata
I20260809 22:48:14.195564 12414 catalog_manager.cc:4841] TS 8c9f7e54191846bc9ba4e5c8af2aba90 (127.24.136.1:45301): tablet 705aa0b6128d4b718c491610333feaca (table test-table [id=14bacbabaab948f0bec89c2d2592e9a3]) successfully deleted
I20260809 22:48:14.195700 12417 catalog_manager.cc:4841] TS b03353283bc1450c914bf953c840e3a5 (127.24.136.2:43663): tablet 705aa0b6128d4b718c491610333feaca (table test-table [id=14bacbabaab948f0bec89c2d2592e9a3]) successfully deleted
I20260809 22:48:14.200352 13345 ts_tablet_manager.cc:1918] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:14.200572 13345 log.cc:1196] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/705aa0b6128d4b718c491610333feaca
I20260809 22:48:14.201215 13291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.171873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.24.136.1:45301)
0809 22:48:14.172100 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:14.172107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:14.181708 (+ 9601us) server_negotiation.cc:407] Connection header received
0809 22:48:14.182046 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:14.182052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:14.182166 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:14.182247 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:14.184406 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.185556 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.187592 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.188262 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.188384 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:14.194636 (+ 6252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:14.194654 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:14.194672 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:14.194734 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:14.198021 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:14.198859 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:14.198868 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:14.198870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:14.199020 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:14.199491 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:14.199497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:14.199500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:14.199895 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:14.200234 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:14.200564 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 22:48:14.200840 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 22:48:14.201768 13345 ts_tablet_manager.cc:1939] T 705aa0b6128d4b718c491610333feaca P 58729dc203d7471ca95c0da16b8edf76: Deleting consensus metadata
I20260809 22:48:14.201936 12797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:14.204141 12413 catalog_manager.cc:4841] TS 58729dc203d7471ca95c0da16b8edf76 (127.24.136.3:33887): tablet 705aa0b6128d4b718c491610333feaca (table test-table [id=14bacbabaab948f0bec89c2d2592e9a3]) successfully deleted
I20260809 22:48:14.313078 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 12569
I20260809 22:48:14.341823 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 12818
Aug 09 22:48:14 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:48:14.360656 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.342729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45140 (local address 127.24.136.2:43663)
0809 22:48:14.342969 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:14.342976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:14.343002 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:48:14.343136 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:14.343143 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:14.343233 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:14.343351 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:14.344882 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.346145 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.347686 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.348349 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.348495 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:14.353794 (+ 5299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:14.353816 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:14.353835 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:14.353889 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:14.357099 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:14.358134 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:14.358142 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:14.358145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:14.358306 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:14.358836 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:14.358841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:14.358843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:14.359350 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:14.359790 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:14.360111 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 22:48:14.360320 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 22:48:14.361533 13029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:14.453984 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 12818
I20260809 22:48:14.476681 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 13053
Aug 09 22:48:14 dist-test-slave-pww0 krb5kdc[11578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:48:14.495944 13294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.477504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40792 (local address 127.24.136.3:33887)
0809 22:48:14.477682 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:14.477688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:14.477820 (+ 132us) server_negotiation.cc:407] Connection header received
0809 22:48:14.477991 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:14.477995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:14.478067 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:14.478161 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:14.479667 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.480883 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.482327 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:14.483010 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:14.483166 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:14.487908 (+ 4742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:14.487934 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:14.487962 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:14.488029 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:14.492428 (+ 4399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:14.493326 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:14.493337 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:14.493340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:14.493559 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:14.494032 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:14.494038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:14.494041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:14.494566 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:14.494929 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:14.495192 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 22:48:14.495535 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:48:14.496894 13261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:14.622356 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 13053
I20260809 22:48:14.651211 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 12386
I20260809 22:48:14.652781 12482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:14.767048 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 12386
I20260809 22:48:14.802588 25120 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 22:48:15.150965 25120 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 22:48:15.151302 25120 mini_ranger.cc:66] Stopping Ranger...
I20260809 22:48:15.497954 25120 mini_ranger.cc:68] Stopped Ranger
2026-08-09 22:48:15.498 UTC [11638] LOG: received smart shutdown request
2026-08-09 22:48:15.502 UTC [11638] LOG: background worker "logical replication launcher" (PID 11659) exited with exit code 1
2026-08-09 22:48:15.502 UTC [11651] LOG: shutting down
2026-08-09 22:48:15.675 UTC [11638] LOG: database system is shut down
2026-08-09T22:48:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53921 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:15 dist-test-slave-pww0 krb5kdc[13376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:15 dist-test-slave-pww0 krb5kdc[13376](info): set up 2 sockets
Aug 09 22:48:15 dist-test-slave-pww0 krb5kdc[13376](info): commencing operation
krb5kdc: starting...
W20260809 22:48:17.067950 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:48:17 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.136.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.136.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/rangeradmin_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/rangeradmin_127.24.136.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.136.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.136.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/rangerlookup_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/rangerlookup_127.24.136.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.136.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.136.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/HTTP_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/HTTP_127.24.136.20@KRBTEST.COM.keytab.
I20260809 22:48:17.470120 25120 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknxBkGy/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/postgres -l logfile start
2026-08-09 22:48:20.012 UTC [13427] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 22:48:20.012 UTC [13427] LOG: listening on IPv4 address "127.24.136.20", port 35879
2026-08-09 22:48:20.015 UTC [13427] LOG: listening on Unix socket "/tmp/.s.PGSQL.35879"
2026-08-09 22:48:20.039 UTC [13434] LOG: database system was shut down at 2026-08-09 22:48:18 UTC
2026-08-09 22:48:20.043 UTC [13427] LOG: database system is ready to accept connections
I20260809 22:48:21.176244 25120 mini_postgres.cc:96] Postgres bound to 35879
2026-08-09 22:48:21.199 UTC [13452] FATAL: database "slave" does not exist
127.24.136.20:35879 - accepting connections
I20260809 22:48:21.199779 25120 mini_ranger.cc:161] Starting Ranger...
I20260809 22:48:21.232368 25120 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 22:48:21.728317 25120 mini_ranger.cc:87] Created ranger Postgres database
I20260809 22:48:21.728437 25120 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-admin
2026-08-09 22:48:22,049 [I] DB FLAVOR :POSTGRES
2026-08-09 22:48:22,050 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:48:22,050 [I] Checking connection..
2026-08-09 22:48:22,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 22:48:22,328 [I] Checking connection passed.
2026-08-09 22:48:22,328 [I] --------- Verifying version history table ---------
2026-08-09 22:48:22,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:48:22,609 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:48:22,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:48:22,872 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:48:22,872 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 22:48:22,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 22:48:23.123 UTC [13555] WARNING: there is no transaction in progress
2026-08-09 22:48:23,135 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 22:48:23,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:48:23,421 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 22:48:23,421 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:48:23,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 22:48:23,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 22:48:23,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pww0.c.gcp-upstream.internal','N') ;"
2026-08-09 22:48:24,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:48:24,458 [I] Table x_portal_user does not exist in database ranger
2026-08-09 22:48:24,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:48:24,732 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 22:48:24,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:48:25,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:48:25,285 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 22:48:25,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 22:48:26.287 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.304 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.320 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.336 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.355 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.372 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.429 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.436 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.452 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.464 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.475 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.492 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.502 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.509 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.516 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:26.523 UTC [13737] WARNING: there is no transaction in progress
2026-08-09 22:48:27,238 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 22:48:27,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 22:48:27,506 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 22:48:27,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:48:27,767 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 22:48:27,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:48:28,057 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 22:48:28,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:48:28,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:48:28,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pww0.c.gcp-upstream.internal';"
2026-08-09 22:48:28,821 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 22:48:28,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 22:48:29,085 [I] DB_PATCHES have already been applied
I20260809 22:48:29.091696 25120 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknxBkGy/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 22:48:29.091796 25120 mini_ranger.cc:193] Using host: 127.24.136.20
I20260809 22:48:29.112198 25120 mini_ranger.cc:227] Ranger admin URL: http://127.24.136.20:35917
Aug 09, 2026 10:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 10:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 10:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 10:48:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 10:48:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 22:48:30.424750 25120 mini_ranger.cc:160] Time spent starting Ranger: real 9.225s user 0.002s sys 0.063s
Aug 09, 2026 10:48:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 10:48:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 10:48:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 10:48:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 10:48:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 22:48:53.383525 25120 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.136.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.136.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/rangerkms_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.136.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/rangerkms_127.24.136.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.136.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.136.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.136.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/HTTP_127.24.136.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.136.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/HTTP_127.24.136.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 22:48:53 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315733, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 22:48:53.600647 25120 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 22:48:53.629586 25120 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 22:48:54.446482 25120 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 22:48:54.446606 25120 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-kms
I20260809 22:48:54.446662 25120 mini_ranger_kms.cc:227] Using postgres at 127.24.136.20:35879
2026-08-09 22:48:54,819 [I] DB FLAVOR :POSTGRES
2026-08-09 22:48:54,819 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:48:54,820 [I] Checking connection
2026-08-09 22:48:54,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 22:48:55,095 [I] connection success
2026-08-09 22:48:55,096 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 22:48:55,096 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 22:48:55,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 22:48:55,372 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 22:48:55,373 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:48:55,373 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 22:48:55,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.136.20:35879/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 22:48:55,650 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 22:48:56.225811 25120 mini_ranger_kms.cc:310] Created kms service
I20260809 22:48:56.412027 25120 mini_ranger_kms.cc:326] Created kudu user
I20260809 22:48:56.459076 25120 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 22:48:56 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315733, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:48:56.861078 25120 mini_ranger_kms.cc:384] Added ranger policy
I20260809 22:48:56.861442 25120 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknxBkGy/build/asan/bin/postgresql.jar:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknxBkGy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 22:48:56.861516 25120 mini_ranger_kms.cc:240] Using host: 127.24.136.20
I20260809 22:48:56.889635 25120 mini_ranger_kms.cc:276] Ranger KMS PID: 14040
I20260809 22:48:56.889763 25120 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.136.20:53043
Aug 09, 2026 10:48:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:48:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:48:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 10:48:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 10:48:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 10:48:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 22:48:59 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315739, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.136.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 22:49:00 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315739, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.136.20@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:00.162099 25120 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.561s user 0.084s sys 0.344s
I20260809 22:49:00.162349 25120 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 22:49:00.162506 25120 mini_ranger_kms.cc:401] 127.24.136.20:53043/kms/v1/keys
Aug 09 22:49:00 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315739, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.136.20@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
Aug 09 22:49:02 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:49:02.973635 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38169
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38169
--ranger_config_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:53043/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:03.346509 14124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:03.346829 14124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:03.346962 14124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:03.354434 14124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:03.354569 14124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:03.354619 14124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:03.354699 14124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:03.354765 14124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:03.365229 14124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.136.20:53043/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38169
--ranger_config_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:03.366724 14124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:03.368301 14124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:03.380959 14141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:03.380998 14138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:03.381029 14134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:03.381762 14124 server_base.cc:1034] running on GCE node
I20260809 22:49:03.382623 14124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:03.384244 14124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:03.385488 14124 hybrid_clock.cc:648] HybridClock initialized: now 1786315743385437 us; error 37 us; skew 500 ppm
I20260809 22:49:03.385881 14124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:03 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315743, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:03.393625 14124 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:49:03.395421 14124 webserver.cc:460] Webserver started at http://127.24.136.62:36771/ using document root <none> and password file <none>
I20260809 22:49:03.396203 14124 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:03.396310 14124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:03.396597 14124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:49:03 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315743, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:03.456060 14124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "9d980b3ee93f4626b1e06d0ecb7e875e"
format_stamp: "Formatted at 2026-08-09 22:49:03 on dist-test-slave-pww0"
server_key: "a2861c0abeb73c241decc47ce4b4558b"
server_key_iv: "65252157444e2a0c56da159ea9ab293f"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:03.456975 14124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "9d980b3ee93f4626b1e06d0ecb7e875e"
format_stamp: "Formatted at 2026-08-09 22:49:03 on dist-test-slave-pww0"
server_key: "a2861c0abeb73c241decc47ce4b4558b"
server_key_iv: "65252157444e2a0c56da159ea9ab293f"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:03.462625 14124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:49:03.482161 14155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:03.483927 14124 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:49:03.484133 14124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "9d980b3ee93f4626b1e06d0ecb7e875e"
format_stamp: "Formatted at 2026-08-09 22:49:03 on dist-test-slave-pww0"
server_key: "a2861c0abeb73c241decc47ce4b4558b"
server_key_iv: "65252157444e2a0c56da159ea9ab293f"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:03.484371 14124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:03.514465 14124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:03.518800 14124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:03.519126 14124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:03.540397 14124 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38169
I20260809 22:49:03.540421 14244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38169 every 8 connection(s)
I20260809 22:49:03.541821 14124 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:03.546155 14247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:03.546319 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 14124
I20260809 22:49:03.546525 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:49:03.547178 25120 external_mini_cluster.cc:1428] Setting key 88ac3620949d160e37c6ee56ce9e7fa1
I20260809 22:49:03.560606 14247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Bootstrap starting.
Aug 09 22:49:03 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:03.563931 14247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:03.565335 14247 log.cc:824] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:03.569123 14247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: No bootstrap required, opened a new log
I20260809 22:49:03.571897 14253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:03.549438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.24.136.62:38169)
0809 22:49:03.550437 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:03.550459 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:03.550824 (+ 365us) server_negotiation.cc:407] Connection header received
0809 22:49:03.552671 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:03.552705 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:03.553279 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:03.553771 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:03.554956 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:03.557011 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:03.558322 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:03.559375 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:03.559576 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:03.563379 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:03.563390 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:03.563409 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:03.563486 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:03.566985 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:03.567906 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:03.567915 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:03.567924 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:03.568146 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:03.568556 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:03.568560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:03.568563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:03.569317 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:03.569610 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:03.570011 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 22:49:03.570475 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":414,"threads_started":1}
I20260809 22:49:03.578096 14247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } }
I20260809 22:49:03.578475 14247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:03.578550 14247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d980b3ee93f4626b1e06d0ecb7e875e, State: Initialized, Role: FOLLOWER
I20260809 22:49:03.579071 14247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } }
I20260809 22:49:03.579282 14247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:03.579381 14247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:03.579541 14247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:03.581689 14247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } }
I20260809 22:49:03.582178 14247 leader_election.cc:302] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d980b3ee93f4626b1e06d0ecb7e875e; no voters:
I20260809 22:49:03.582741 14247 leader_election.cc:288] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:03.582870 14258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:03.583643 14258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 1 LEADER]: Becoming Leader. State: Replica: 9d980b3ee93f4626b1e06d0ecb7e875e, State: Running, Role: LEADER
I20260809 22:49:03.584194 14258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } }
I20260809 22:49:03.584781 14247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:03.587245 14260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d980b3ee93f4626b1e06d0ecb7e875e. Latest consensus state: current_term: 1 leader_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } } }
I20260809 22:49:03.587703 14260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:03.587282 14259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } } }
I20260809 22:49:03.588256 14259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:03.589282 14270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:03.596029 14270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:03.596807 14247 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 22:49:03.621845 14270 catalog_manager.cc:1347] Generated new cluster ID: 6c876cc2ca05448a97f7272df750df74
I20260809 22:49:03.622000 14270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:03.641096 14270 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:03.642798 14270 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:03.649938 14270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Generated new TSK 0
I20260809 22:49:03.650990 14270 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:49:04.445756 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:53043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38169
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:49:04 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315744, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 22:49:04.908360 14318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:04.908720 14318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:04.908872 14318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:04.917893 14318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:04.918025 14318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:04.918180 14318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:49:04.929760 14318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.136.20:53043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:04.931267 14318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:04.932729 14318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:04.952580 14331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:04.963721 14332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:04.965788 14336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:04.966440 14318 server_base.cc:1034] running on GCE node
I20260809 22:49:04.967365 14318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:04.968549 14318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:04.969805 14318 hybrid_clock.cc:648] HybridClock initialized: now 1786315744969729 us; error 73 us; skew 500 ppm
I20260809 22:49:04.970181 14318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:04 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315744, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:04.973878 14318 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:49:04.975747 14318 webserver.cc:460] Webserver started at http://127.24.136.1:38171/ using document root <none> and password file <none>
I20260809 22:49:04.976557 14318 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:04.976689 14318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:04.977028 14318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:49:04 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315744, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:04.996337 14318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "1cbbebb6f1b748e8b451e0606f7c1879"
format_stamp: "Formatted at 2026-08-09 22:49:04 on dist-test-slave-pww0"
server_key: "d20c8ffa56c9eb856823c6e99cf7d6ec"
server_key_iv: "e60f399f05a0a808939ff910bfca9973"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:04.997397 14318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "1cbbebb6f1b748e8b451e0606f7c1879"
format_stamp: "Formatted at 2026-08-09 22:49:04 on dist-test-slave-pww0"
server_key: "d20c8ffa56c9eb856823c6e99cf7d6ec"
server_key_iv: "e60f399f05a0a808939ff910bfca9973"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:05.012369 14318 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.010s sys 0.005s
I20260809 22:49:05.038089 14349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:05.040354 14318 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.005s sys 0.000s
I20260809 22:49:05.040568 14318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "1cbbebb6f1b748e8b451e0606f7c1879"
format_stamp: "Formatted at 2026-08-09 22:49:04 on dist-test-slave-pww0"
server_key: "d20c8ffa56c9eb856823c6e99cf7d6ec"
server_key_iv: "e60f399f05a0a808939ff910bfca9973"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:05.040812 14318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:05.076017 14318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:05.081745 14318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:05.082113 14318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:05.083540 14318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:05.085847 14318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:05.085990 14318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:49:05.086144 14318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:05.086226 14318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:49:05 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315744, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:05.310202 14318 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:36179
I20260809 22:49:05.310227 14549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:36179 every 8 connection(s)
I20260809 22:49:05.311772 14318 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:49:05.314272 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 14318
I20260809 22:49:05.314500 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:49:05.315349 25120 external_mini_cluster.cc:1428] Setting key f826a5d07ce3c1af4209ecc3b6ddfcc6
Aug 09 22:49:05 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315744, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:05.372386 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:05.316507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:45309 (local address 127.24.136.62:38169)
0809 22:49:05.331863 (+ 15356us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:05.331871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:05.338522 (+ 6651us) server_negotiation.cc:407] Connection header received
0809 22:49:05.341614 (+ 3092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:05.341621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:05.341744 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:05.341869 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:05.344408 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:05.345533 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:05.356573 (+ 11040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:05.357318 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:05.357448 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:05.362551 (+ 5103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:05.362559 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:05.362563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:05.362610 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:05.366406 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:05.367655 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:05.367661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:05.367665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:05.367862 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:05.368517 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:05.368522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:05.368526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:05.368927 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:05.369090 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:05.371673 (+ 2583us) server_negotiation.cc:299] Negotiation successful
0809 22:49:05.371965 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15195,"thread_start_us":15364,"threads_started":1}
I20260809 22:49:05.379994 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:05.315638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38169 (local address 127.24.136.1:45309)
0809 22:49:05.338214 (+ 22576us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:05.338314 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:05.341063 (+ 2749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:05.342418 (+ 1355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:05.342432 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:05.343130 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:05.344128 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:05.344158 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:05.345767 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:05.345773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:05.356233 (+ 10460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:05.356279 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:05.357902 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:05.357913 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:05.358315 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:05.359234 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:05.359286 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:05.362184 (+ 2898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:05.366657 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:05.366669 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:05.366691 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:05.367398 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:05.368023 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:05.368028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:05.368032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:05.368357 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:05.369177 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:05.369187 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:05.369734 (+ 547us) client_negotiation.cc:769] Sending connection context
0809 22:49:05.370147 (+ 413us) client_negotiation.cc:240] Negotiation successful
0809 22:49:05.370845 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22157,"thread_start_us":24393,"threads_started":1}
I20260809 22:49:05.382584 14550 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38169
I20260809 22:49:05.382970 14550 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:05.384136 14550 heartbeater.cc:511] Master 127.24.136.62:38169 requested a full tablet report, sending...
I20260809 22:49:05.387172 14180 ts_manager.cc:194] Registered new tserver with Master: 1cbbebb6f1b748e8b451e0606f7c1879 (127.24.136.1:36179)
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
I20260809 22:49:05.389452 14180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:45309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
W20260809 22:49:05.437425 25120 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.136.2: real 0.119s user 0.000s sys 0.025s
I20260809 22:49:05.438452 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:53043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38169
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:49:05 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315744, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
W20260809 22:49:05.973745 14561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:05.974273 14561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:05.974447 14561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:05.986236 14561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:05.986491 14561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:05.986683 14561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:49:06.002336 14561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.136.20:53043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:06.004117 14561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:06.005859 14561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:06.023825 14584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:06.024241 14585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:06.032282 14587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:06.033485 14561 server_base.cc:1034] running on GCE node
I20260809 22:49:06.034359 14561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:06.035837 14561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:06.037140 14561 hybrid_clock.cc:648] HybridClock initialized: now 1786315746037062 us; error 63 us; skew 500 ppm
I20260809 22:49:06.037563 14561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:06 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315746, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:06.041920 14561 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:49:06.044183 14561 webserver.cc:460] Webserver started at http://127.24.136.2:39003/ using document root <none> and password file <none>
I20260809 22:49:06.045188 14561 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:06.045481 14561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:06.045876 14561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:49:06 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315746, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:06.067991 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "234fb959b117472fa47182571b273420"
format_stamp: "Formatted at 2026-08-09 22:49:06 on dist-test-slave-pww0"
server_key: "63158e6e52778ffb6b9cb87ee3671713"
server_key_iv: "ae1e51bd1a990c0b37a00f76533b6ae7"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:06.069222 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "234fb959b117472fa47182571b273420"
format_stamp: "Formatted at 2026-08-09 22:49:06 on dist-test-slave-pww0"
server_key: "63158e6e52778ffb6b9cb87ee3671713"
server_key_iv: "ae1e51bd1a990c0b37a00f76533b6ae7"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:06.077234 14561 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260809 22:49:06.101310 14603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:06.103381 14561 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:49:06.103648 14561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "234fb959b117472fa47182571b273420"
format_stamp: "Formatted at 2026-08-09 22:49:06 on dist-test-slave-pww0"
server_key: "63158e6e52778ffb6b9cb87ee3671713"
server_key_iv: "ae1e51bd1a990c0b37a00f76533b6ae7"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:06.103922 14561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:06.140352 14561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:06.146400 14561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:06.146770 14561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:06.148252 14561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:06.150328 14561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:06.150563 14561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:49:06.150755 14561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:06.150842 14561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:06.193287 14561 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:33049
I20260809 22:49:06.193333 14801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:33049 every 8 connection(s)
I20260809 22:49:06.194978 14561 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:49:06.199597 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 14561
I20260809 22:49:06.199800 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:49:06.200435 25120 external_mini_cluster.cc:1428] Setting key 493fa444785da5d141b69254c94d3d39
Aug 09 22:49:06 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315746, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:06.231753 14807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:06.201215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:60785 (local address 127.24.136.62:38169)
0809 22:49:06.208019 (+ 6804us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:06.208026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:06.211768 (+ 3742us) server_negotiation.cc:407] Connection header received
0809 22:49:06.211852 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:06.211857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:06.211953 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:06.212057 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:06.214413 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.215617 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.218192 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.218811 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.218915 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:06.223738 (+ 4823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:06.223745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:06.223750 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:06.223798 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:06.227406 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:06.228585 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:06.228591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:06.228594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:06.228767 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:06.229356 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:06.229361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:06.229365 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:06.229747 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:06.229879 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:06.230965 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0809 22:49:06.231231 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6647,"thread_start_us":6496,"threads_started":1}
I20260809 22:49:06.234493 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:06.198663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38169 (local address 127.24.136.2:60785)
0809 22:49:06.208715 (+ 10052us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:06.208791 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:06.211271 (+ 2480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:06.212565 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:06.212578 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:06.213241 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:06.214164 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:06.214194 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.215860 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.215865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:06.217982 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:06.218000 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.219291 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.219298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:06.219643 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:06.220472 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:06.220509 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:06.223366 (+ 2857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:06.227691 (+ 4325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:06.227702 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:06.227725 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:06.228419 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:06.228928 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:06.228932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:06.228935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:06.229208 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:06.229992 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:06.230001 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:06.230509 (+ 508us) client_negotiation.cc:769] Sending connection context
0809 22:49:06.232346 (+ 1837us) client_negotiation.cc:240] Negotiation successful
0809 22:49:06.232942 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9684,"thread_start_us":9431,"threads_started":1}
I20260809 22:49:06.237003 14802 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38169
I20260809 22:49:06.237362 14802 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:06.238310 14802 heartbeater.cc:511] Master 127.24.136.62:38169 requested a full tablet report, sending...
I20260809 22:49:06.240231 14180 ts_manager.cc:194] Registered new tserver with Master: 234fb959b117472fa47182571b273420 (127.24.136.2:33049)
I20260809 22:49:06.241403 14180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:60785
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:49:06.289036 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:53043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38169
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:06.393950 14550 heartbeater.cc:503] Master 127.24.136.62:38169 was elected leader, sending a full tablet report...
W20260809 22:49:06.650038 14811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:06.650383 14811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:06.650484 14811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:06.658069 14811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:06.658200 14811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:06.658315 14811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:49:06.668802 14811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.136.20:53043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:06.670149 14811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:06.671591 14811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:06.685495 14816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:06.685390 14820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:06.685420 14817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:06.686831 14811 server_base.cc:1034] running on GCE node
I20260809 22:49:06.687588 14811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:06.688676 14811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:06.689882 14811 hybrid_clock.cc:648] HybridClock initialized: now 1786315746689833 us; error 44 us; skew 500 ppm
I20260809 22:49:06.690186 14811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:06 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315746, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:06.693475 14811 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:49:06.695056 14811 webserver.cc:460] Webserver started at http://127.24.136.3:36131/ using document root <none> and password file <none>
I20260809 22:49:06.695865 14811 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:06.695995 14811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:06.696306 14811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:49:06 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315746, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:06.715761 14811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff"
format_stamp: "Formatted at 2026-08-09 22:49:06 on dist-test-slave-pww0"
server_key: "db72a348ee5e4a36c88121839498c537"
server_key_iv: "a312edae4e509f8d7c7e782121d3ba8e"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:06.717020 14811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff"
format_stamp: "Formatted at 2026-08-09 22:49:06 on dist-test-slave-pww0"
server_key: "db72a348ee5e4a36c88121839498c537"
server_key_iv: "a312edae4e509f8d7c7e782121d3ba8e"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:06.724895 14811 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260809 22:49:06.743978 14830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:06.745864 14811 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:49:06.746163 14811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff"
format_stamp: "Formatted at 2026-08-09 22:49:06 on dist-test-slave-pww0"
server_key: "db72a348ee5e4a36c88121839498c537"
server_key_iv: "a312edae4e509f8d7c7e782121d3ba8e"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:06.746460 14811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:06.777428 14811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:06.781932 14811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:06.782225 14811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:06.783385 14811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:06.784937 14811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:06.785022 14811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:06.785138 14811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:06.785183 14811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:06.817715 14811 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:44245
I20260809 22:49:06.817772 15018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:44245 every 8 connection(s)
I20260809 22:49:06.819187 14811 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:49:06.820257 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 14811
I20260809 22:49:06.820453 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:49:06.820901 25120 external_mini_cluster.cc:1428] Setting key f1588962c474601ce2ab0ba9beb2ef1d
Aug 09 22:49:06 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315746, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:06.840099 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:06.822065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:36691 (local address 127.24.136.62:38169)
0809 22:49:06.822561 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:06.822567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:06.823796 (+ 1229us) server_negotiation.cc:407] Connection header received
0809 22:49:06.825832 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:06.825837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:06.825911 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:06.826009 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:06.827547 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.828292 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.829935 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.830415 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.830515 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:06.834059 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:06.834065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:06.834069 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:06.834133 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:06.836866 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:06.837647 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:06.837653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:06.837655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:06.837792 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:06.838270 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:06.838274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:06.838276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:06.838567 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:06.838707 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:06.839501 (+ 794us) server_negotiation.cc:299] Negotiation successful
0809 22:49:06.839748 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":224,"threads_started":1}
I20260809 22:49:06.841118 15025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:06.822665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38169 (local address 127.24.136.3:36691)
0809 22:49:06.823612 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:06.823666 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:06.825505 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:06.826217 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:06.826225 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:06.826680 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:06.827335 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:06.827355 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.828445 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.828449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:06.829783 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:06.829805 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.830614 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.830619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:06.830850 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:06.831577 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:06.831606 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:06.833798 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:06.837024 (+ 3226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:06.837035 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:06.837053 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:06.837522 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:06.837909 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:06.837914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:06.837917 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:06.838147 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:06.838702 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:06.838712 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:06.839092 (+ 380us) client_negotiation.cc:769] Sending connection context
0809 22:49:06.839361 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 22:49:06.839965 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":359,"threads_started":1}
I20260809 22:49:06.842605 15020 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38169
I20260809 22:49:06.842919 15020 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:06.843732 15020 heartbeater.cc:511] Master 127.24.136.62:38169 requested a full tablet report, sending...
I20260809 22:49:06.845455 14180 ts_manager.cc:194] Registered new tserver with Master: b0daa246d2bc41d09bca2ccdcf2b46ff (127.24.136.3:44245)
I20260809 22:49:06.846408 14180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:36691
I20260809 22:49:06.849781 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:49:06.851498 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 14318
Aug 09 22:49:06 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:49:06.868988 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:06.852636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.24.136.1:36179)
0809 22:49:06.853258 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:06.853267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:06.853302 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:49:06.853372 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:06.853380 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:06.853616 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:06.853757 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:06.855320 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.856424 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.857863 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.858445 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.858546 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:06.863150 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:06.863164 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:06.863198 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:06.863248 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:06.866099 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:06.866925 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:06.866933 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:06.866935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:06.867099 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:06.867510 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:06.867515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:06.867518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:06.867895 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:06.868136 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:06.868478 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 22:49:06.868703 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":282,"threads_started":1}
I20260809 22:49:06.870383 14544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:06.955024 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 14318
I20260809 22:49:06.974431 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 14561
Aug 09 22:49:06 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:49:06.996542 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:06.975439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.24.136.2:33049)
0809 22:49:06.976023 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:06.976031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:06.976060 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:49:06.976124 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:06.976131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:06.976356 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:06.976497 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:06.977946 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.978950 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.980436 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:06.981036 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:06.981147 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:06.990543 (+ 9396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:06.990568 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:06.990586 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:06.990633 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:06.993811 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:06.994625 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:06.994633 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:06.994635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:06.994783 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:06.995097 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:06.995100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:06.995102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:06.995515 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:06.995772 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:06.996010 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 22:49:06.996239 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":283,"threads_started":1}
I20260809 22:49:06.997897 14797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:07.082825 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 14561
I20260809 22:49:07.104157 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 14811
Aug 09 22:49:07 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:49:07.122004 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:07.105228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35484 (local address 127.24.136.3:44245)
0809 22:49:07.105878 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:07.105886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:07.105917 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:49:07.105980 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:07.105986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:07.106209 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:07.106361 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:07.107664 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:07.108738 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:07.110087 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:07.110682 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:07.110787 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:07.115523 (+ 4736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:07.115541 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:07.115560 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:07.115606 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:07.118914 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:07.119872 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:07.119881 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:07.119884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:07.120030 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:07.120395 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:07.120398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:07.120400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:07.120782 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:07.121033 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:07.121408 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 22:49:07.121633 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":338,"threads_started":1}
I20260809 22:49:07.123657 15013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:07.212131 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 14811
I20260809 22:49:07.234068 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 14124
I20260809 22:49:07.235355 14238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:07.343550 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 14124
I20260809 22:49:07.391180 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38169
--webserver_interface=127.24.136.62
--webserver_port=36771
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38169
--ranger_config_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:53043/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.136.20:53043/kms with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:07.739989 15041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:07.740314 15041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:07.740461 15041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:07.747856 15041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:07.747977 15041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:07.748050 15041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:07.748137 15041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:07.748206 15041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:07.758878 15041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.136.20:53043/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38169
--ranger_config_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=36771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:07.760346 15041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:07.761727 15041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:07.774262 15053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:07.774225 15047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:07.774569 15049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:07.775400 15041 server_base.cc:1034] running on GCE node
I20260809 22:49:07.776280 15041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:07.777884 15041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:07.779093 15041 hybrid_clock.cc:648] HybridClock initialized: now 1786315747779040 us; error 51 us; skew 500 ppm
I20260809 22:49:07.779439 15041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:07 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315747, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:07.782939 15041 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:49:07.784638 15041 webserver.cc:460] Webserver started at http://127.24.136.62:36771/ using document root <none> and password file <none>
I20260809 22:49:07.785362 15041 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:07.785518 15041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:49:07 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315747, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:07.830188 15041 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 22:49:07.833635 15069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:07.835104 15041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:49:07.835290 15041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "9d980b3ee93f4626b1e06d0ecb7e875e"
format_stamp: "Formatted at 2026-08-09 22:49:03 on dist-test-slave-pww0"
server_key: "a2861c0abeb73c241decc47ce4b4558b"
server_key_iv: "65252157444e2a0c56da159ea9ab293f"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:07.836309 15041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:07.851931 15041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:07.855969 15041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:07.856256 15041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:07.874481 15041 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38169
I20260809 22:49:07.874522 15169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38169 every 8 connection(s)
I20260809 22:49:07.876312 15041 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:07.879685 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 15041
I20260809 22:49:07.880806 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:53043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:36179
--local_ip_for_outbound_sockets=127.24.136.1
--tserver_master_addrs=127.24.136.62:38169
--webserver_port=38171
--webserver_interface=127.24.136.1
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.136.20:53043/kms with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:07.885462 15170 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:49:07.890331 15170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Bootstrap starting.
I20260809 22:49:07.910997 15170 log.cc:824] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:07.924631 15170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:49:07.925375 15170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Bootstrap complete.
I20260809 22:49:07.939061 15170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } }
I20260809 22:49:07.940518 15170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d980b3ee93f4626b1e06d0ecb7e875e, State: Initialized, Role: FOLLOWER
I20260809 22:49:07.941437 15170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } }
I20260809 22:49:07.941778 15170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:07.941911 15170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:07.942095 15170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:49:07.948203 15170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } }
I20260809 22:49:07.948984 15170 leader_election.cc:302] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d980b3ee93f4626b1e06d0ecb7e875e; no voters:
I20260809 22:49:07.949836 15170 leader_election.cc:288] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:49:07.950134 15177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:49:07.951207 15177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [term 2 LEADER]: Becoming Leader. State: Replica: 9d980b3ee93f4626b1e06d0ecb7e875e, State: Running, Role: LEADER
I20260809 22:49:07.951875 15177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } }
I20260809 22:49:07.952643 15170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:07.954449 15181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d980b3ee93f4626b1e06d0ecb7e875e. Latest consensus state: current_term: 2 leader_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } } }
I20260809 22:49:07.954444 15178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d980b3ee93f4626b1e06d0ecb7e875e" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38169 } } }
I20260809 22:49:07.954880 15181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:07.954885 15178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:07.956009 15187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:07.972303 15187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:07.974020 15187 catalog_manager.cc:1259] Loaded cluster ID: 6c876cc2ca05448a97f7272df750df74
I20260809 22:49:07.974251 15187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:07.978761 15187 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:07.980541 15187 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Loaded TSK: 0
I20260809 22:49:07.982625 15187 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:07.989837 15170 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 22:49:08.362187 15174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:08.362696 15174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:08.362917 15174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:08.376194 15174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:08.376473 15174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:08.376745 15174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:49:08.393213 15174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.136.20:53043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:36179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=38171
--tserver_master_addrs=127.24.136.62:38169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:08.395525 15174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:08.397464 15174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:08.424795 15236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:08.429106 15240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:08.431843 15234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:08.436035 15174 server_base.cc:1034] running on GCE node
I20260809 22:49:08.437079 15174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:08.438633 15174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:08.439932 15174 hybrid_clock.cc:648] HybridClock initialized: now 1786315748439851 us; error 64 us; skew 500 ppm
I20260809 22:49:08.440410 15174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:08 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315748, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:08.447661 15174 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:49:08.463959 15174 webserver.cc:460] Webserver started at http://127.24.136.1:38171/ using document root <none> and password file <none>
I20260809 22:49:08.465036 15174 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:08.465305 15174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:49:08 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315748, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:08.493384 15174 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:49:08.497406 15253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:08.499333 15174 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:49:08.499576 15174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "1cbbebb6f1b748e8b451e0606f7c1879"
format_stamp: "Formatted at 2026-08-09 22:49:04 on dist-test-slave-pww0"
server_key: "d20c8ffa56c9eb856823c6e99cf7d6ec"
server_key_iv: "e60f399f05a0a808939ff910bfca9973"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:08.501046 15174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:08.527592 15174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:08.533736 15174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:08.534096 15174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:08.535545 15174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:08.551664 15174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:08.551904 15174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:08.552114 15174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:08.552284 15174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:08.592165 15174 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:36179
I20260809 22:49:08.592231 15441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:36179 every 8 connection(s)
I20260809 22:49:08.594374 15174 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:49:08.596400 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 15174
I20260809 22:49:08.597259 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:53043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:33049
--local_ip_for_outbound_sockets=127.24.136.2
--tserver_master_addrs=127.24.136.62:38169
--webserver_port=39003
--webserver_interface=127.24.136.2
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.136.20:53043/kms with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:49:08 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315748, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:08.637892 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:08.602455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:37775 (local address 127.24.136.62:38169)
0809 22:49:08.610641 (+ 8186us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:08.610664 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:08.610726 (+ 62us) server_negotiation.cc:407] Connection header received
0809 22:49:08.611196 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:08.611246 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:08.611840 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:08.612374 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:08.614960 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:08.616982 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:08.619685 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:08.620849 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:08.621053 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:08.626740 (+ 5687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:08.626755 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:08.626781 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:08.626846 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:08.630882 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:08.632494 (+ 1612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:08.632519 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:08.632530 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:08.632793 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:08.633478 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:08.633484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:08.633488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:08.634269 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:08.634485 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:08.635809 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0809 22:49:08.636340 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7843,"thread_start_us":7965,"threads_started":1}
I20260809 22:49:08.639608 15449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:08.602606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38169 (local address 127.24.136.1:37775)
0809 22:49:08.604190 (+ 1584us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:08.604251 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:08.606733 (+ 2482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:08.613143 (+ 6410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:08.613154 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:08.613835 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:08.614664 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:08.614692 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:08.617261 (+ 2569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:08.617267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:08.619417 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:08.619437 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:08.621591 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:08.621600 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:08.621905 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:08.623294 (+ 1389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:08.623338 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:08.626316 (+ 2978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:08.631089 (+ 4773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:08.631101 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:08.631123 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:08.632001 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:08.632987 (+ 986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:08.632993 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:08.632996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:08.633315 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:08.634610 (+ 1295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:08.634619 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:08.635101 (+ 482us) client_negotiation.cc:769] Sending connection context
0809 22:49:08.637420 (+ 2319us) client_negotiation.cc:240] Negotiation successful
0809 22:49:08.638095 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1259,"thread_start_us":994,"threads_started":1}
I20260809 22:49:08.643637 15442 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38169
I20260809 22:49:08.644008 15442 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:08.644794 15442 heartbeater.cc:511] Master 127.24.136.62:38169 requested a full tablet report, sending...
W20260809 22:49:08.648747 15442 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:38169 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:49:08.649035 15442 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:08.649407 15442 heartbeater.cc:511] Master 127.24.136.62:38169 requested a full tablet report, sending...
I20260809 22:49:08.650985 15442 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:08.651360 15442 heartbeater.cc:511] Master 127.24.136.62:38169 requested a full tablet report, sending...
W20260809 22:49:08.652802 15442 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:49:09.027284 15451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:09.027696 15451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:09.027859 15451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:09.036547 15451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:09.036727 15451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:09.036988 15451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:49:09.048637 15451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.136.20:53043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:33049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=39003
--tserver_master_addrs=127.24.136.62:38169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:09.050028 15451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:09.051724 15451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:09.076843 15476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:09.076823 15478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:09.077705 15474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:09.077980 15451 server_base.cc:1034] running on GCE node
I20260809 22:49:09.078641 15451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:09.079820 15451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:09.081048 15451 hybrid_clock.cc:648] HybridClock initialized: now 1786315749080976 us; error 63 us; skew 500 ppm
I20260809 22:49:09.081379 15451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:09 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:09.084914 15451 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:49:09.086642 15451 webserver.cc:460] Webserver started at http://127.24.136.2:39003/ using document root <none> and password file <none>
I20260809 22:49:09.087621 15451 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:09.087779 15451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:49:09 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:09.115274 15451 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 22:49:09.118374 15495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:09.120007 15451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:49:09.120203 15451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "234fb959b117472fa47182571b273420"
format_stamp: "Formatted at 2026-08-09 22:49:06 on dist-test-slave-pww0"
server_key: "63158e6e52778ffb6b9cb87ee3671713"
server_key_iv: "ae1e51bd1a990c0b37a00f76533b6ae7"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:09.121290 15451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:09.170783 15451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:09.177104 15451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:09.177484 15451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:09.180716 15451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:09.182781 15451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:09.182911 15451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:09.183039 15451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:09.183118 15451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:09.276175 15451 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:33049
I20260809 22:49:09.276242 15702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:33049 every 8 connection(s)
I20260809 22:49:09.278263 15451 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:49:09.283643 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 15451
I20260809 22:49:09.284446 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.136.20:53043/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:44245
--local_ip_for_outbound_sockets=127.24.136.3
--tserver_master_addrs=127.24.136.62:38169
--webserver_port=36131
--webserver_interface=127.24.136.3
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:41173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.136.20:53043/kms with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:49:09 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:09.303771 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:09.281120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:45481 (local address 127.24.136.62:38169)
0809 22:49:09.281574 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:09.281578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:09.282926 (+ 1348us) server_negotiation.cc:407] Connection header received
0809 22:49:09.285451 (+ 2525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:09.285459 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:09.285553 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:09.285678 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:09.288242 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:09.289450 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:09.291749 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:09.292453 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:09.292605 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:09.296681 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:09.296686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:09.296689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:09.296732 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:09.300243 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:09.301289 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:09.301295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:09.301300 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:09.301448 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:09.301918 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:09.301924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:09.301927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:09.302223 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:09.302363 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:09.303119 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 22:49:09.303359 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
Aug 09 22:49:09 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:09.304884 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:09.281858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38169 (local address 127.24.136.2:45481)
0809 22:49:09.282749 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:09.282793 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:09.285093 (+ 2300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:09.285974 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:09.285987 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:09.286595 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:09.288000 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:09.288029 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:09.289643 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:09.289649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:09.291567 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:09.291599 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:09.292729 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:09.292737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:09.293015 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:09.293848 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:09.293879 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:09.296431 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:09.300453 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:09.300463 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:09.300485 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:09.301135 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:09.301564 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:09.301569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:09.301571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:09.301791 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:09.302364 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:09.302374 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:09.302735 (+ 361us) client_negotiation.cc:769] Sending connection context
0809 22:49:09.302984 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 22:49:09.303616 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":381,"threads_started":1}
I20260809 22:49:09.306521 15705 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38169
I20260809 22:49:09.306833 15705 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:09.307679 15705 heartbeater.cc:511] Master 127.24.136.62:38169 requested a full tablet report, sending...
I20260809 22:49:09.309787 15105 ts_manager.cc:194] Registered new tserver with Master: 234fb959b117472fa47182571b273420 (127.24.136.2:33049)
I20260809 22:49:09.311929 15105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:45481
I20260809 22:49:09.652957 15442 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:09.653802 15442 heartbeater.cc:511] Master 127.24.136.62:38169 requested a full tablet report, sending...
I20260809 22:49:09.655339 15105 ts_manager.cc:194] Registered new tserver with Master: 1cbbebb6f1b748e8b451e0606f7c1879 (127.24.136.1:36179)
I20260809 22:49:09.656630 15105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:37775
Aug 09 22:49:09 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
W20260809 22:49:09.779856 15715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:09.780458 15715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:09.780637 15715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:09.789723 15715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:09.789914 15715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:09.790114 15715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:49:09.806331 15715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:41173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.136.20:53043/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:44245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=36131
--tserver_master_addrs=127.24.136.62:38169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:09.808380 15715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:09.810158 15715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:09.852375 15729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:09.856707 15726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:09.857484 15733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:09.858248 15715 server_base.cc:1034] running on GCE node
I20260809 22:49:09.859216 15715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:09.883834 15715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:09.887145 15715 hybrid_clock.cc:648] HybridClock initialized: now 1786315749887047 us; error 113 us; skew 500 ppm
I20260809 22:49:09.887709 15715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:09 dist-test-slave-pww0 krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:09.892481 15715 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:49:09.894657 15715 webserver.cc:460] Webserver started at http://127.24.136.3:36131/ using document root <none> and password file <none>
I20260809 22:49:09.895720 15715 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:09.895896 15715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:49:09 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:09.945847 15715 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.005s sys 0.005s
I20260809 22:49:09.964412 15741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:09.966738 15715 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.005s sys 0.000s
I20260809 22:49:09.967072 15715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff"
format_stamp: "Formatted at 2026-08-09 22:49:06 on dist-test-slave-pww0"
server_key: "db72a348ee5e4a36c88121839498c537"
server_key_iv: "a312edae4e509f8d7c7e782121d3ba8e"
server_key_version: "kuduclusterkey@0"
I20260809 22:49:09.968794 15715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:10.010797 15715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:10.016594 15715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:10.016995 15715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:10.024546 15715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:10.032944 15715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:10.033172 15715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:49:10.033342 15715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:10.033461 15715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:49:10 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for HTTP/127.24.136.20@KRBTEST.COM
I20260809 22:49:10.316738 15705 heartbeater.cc:503] Master 127.24.136.62:38169 was elected leader, sending a full tablet report...
I20260809 22:49:10.410364 15715 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:44245
I20260809 22:49:10.410449 15936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:44245 every 8 connection(s)
I20260809 22:49:10.412487 15715 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:49:10.421681 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 15715
Aug 09 22:49:10 dist-test-slave-pww0 krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:10.464815 15948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.422900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.24.136.62:38169)
0809 22:49:10.444125 (+ 21225us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.444132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.445408 (+ 1276us) server_negotiation.cc:407] Connection header received
0809 22:49:10.445472 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.445477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.445564 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.445656 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:10.447874 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.449315 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.451359 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.452199 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.452325 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.457057 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:10.457066 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:10.457071 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:10.457119 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:10.461569 (+ 4450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:10.462655 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:10.462659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:10.462662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:10.462825 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:10.463305 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:10.463310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:10.463313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:10.463709 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:10.463928 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.464286 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.464491 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":21051,"thread_start_us":21276,"threads_started":1}
I20260809 22:49:10.467338 15943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.415856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53535 (local address 127.24.136.62:38169)
0809 22:49:10.437976 (+ 22120us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.437983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.438013 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:49:10.444709 (+ 6696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.444717 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.444849 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.444989 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:10.448008 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.449220 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.451697 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.452407 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.452507 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.457551 (+ 5044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:10.457558 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:10.457563 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:10.457624 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:10.461634 (+ 4010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:10.462612 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:10.462618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:10.462621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:10.462824 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:10.463434 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:10.463440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:10.463444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:10.465317 (+ 1873us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:10.465487 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.466459 (+ 972us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.466703 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":21973,"spinlock_wait_cycles":6912,"thread_start_us":21832,"threads_started":1}
I20260809 22:49:10.468364 15946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.416576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38169 (local address 127.24.136.3:53535)
0809 22:49:10.432707 (+ 16131us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:10.432789 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:10.435348 (+ 2559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:10.445962 (+ 10614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:10.445979 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:10.446704 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:10.447736 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.447772 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.449434 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.449440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.451534 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.451550 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.452856 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.452863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.453183 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.453929 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:10.453964 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:10.457214 (+ 3250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:10.461864 (+ 4650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:10.461876 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:10.461893 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:10.462445 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:10.462986 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:10.462991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:10.462995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:10.463279 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:10.465498 (+ 2219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:10.465509 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:10.465997 (+ 488us) client_negotiation.cc:769] Sending connection context
0809 22:49:10.466346 (+ 349us) client_negotiation.cc:240] Negotiation successful
0809 22:49:10.466950 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15717,"thread_start_us":15431,"threads_started":1}
I20260809 22:49:10.470019 15939 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38169
I20260809 22:49:10.470322 15939 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:10.471061 15939 heartbeater.cc:511] Master 127.24.136.62:38169 requested a full tablet report, sending...
I20260809 22:49:10.472748 15105 ts_manager.cc:194] Registered new tserver with Master: b0daa246d2bc41d09bca2ccdcf2b46ff (127.24.136.3:44245)
I20260809 22:49:10.473551 15105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:53535
I20260809 22:49:10.481343 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:49:10.499575 15943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.485502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.24.136.62:38169)
0809 22:49:10.485751 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.485758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.486182 (+ 424us) server_negotiation.cc:407] Connection header received
0809 22:49:10.486361 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.486368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.486474 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.486600 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:10.488375 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.489137 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.490569 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.491012 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.491132 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.493844 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:10.493849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:10.493853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:10.493892 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:10.496723 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:10.497490 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:10.497495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:10.497497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:10.497643 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:10.498045 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:10.498049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:10.498051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:10.498356 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:10.498600 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.498963 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.499237 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:49:10.505546 15105 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:49:10.513909 15105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:49:10.543493 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.530145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:33049 (local address 127.0.0.1:56736)
0809 22:49:10.530940 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:10.530984 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:10.531175 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:10.535651 (+ 4476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:10.535655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:10.535692 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:10.536026 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.536039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.538566 (+ 2527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.538572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.540741 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.540756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.542662 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.542678 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.542832 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.542860 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:49:10.542913 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 22:49:10.543034 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":1,"thread_start_us":623,"threads_started":1}
I20260809 22:49:10.544588 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.530328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.24.136.2:33049)
0809 22:49:10.535046 (+ 4718us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.535055 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.535083 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:49:10.535182 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.535190 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.535393 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.535677 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:10.536241 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.538384 (+ 2143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.540973 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.542411 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.543785 (+ 1374us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.544032 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.544105 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.544262 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4557,"thread_start_us":4397,"threads_started":1}
I20260809 22:49:10.545068 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.533778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35486 (local address 127.24.136.3:44245)
0809 22:49:10.534346 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.534352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.534379 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:49:10.534485 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.534492 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.534770 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.534967 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:10.535584 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.537203 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.540129 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.541808 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.541938 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.542202 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.543161 (+ 959us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.543354 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":579,"threads_started":1}
I20260809 22:49:10.545210 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.533201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44245 (local address 127.0.0.1:35486)
0809 22:49:10.533747 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:10.533765 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:10.533856 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:10.534986 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:10.534993 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:10.535022 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:10.535391 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.535402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.537366 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.537370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.539931 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.539947 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.542374 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.542380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.542601 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.542667 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 22:49:10.542802 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 22:49:10.543028 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":25,"thread_start_us":279,"threads_started":1}
I20260809 22:49:10.545819 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.530295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36179 (local address 127.0.0.1:48170)
0809 22:49:10.531935 (+ 1640us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:10.531955 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:10.532083 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:10.536552 (+ 4469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:10.536557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:10.536581 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:10.536886 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.536896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.539626 (+ 2730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.539632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.543111 (+ 3479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.543127 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.544971 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.544977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.545160 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.545198 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:49:10.545286 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:49:10.545443 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1507,"mutex_wait_us":3,"thread_start_us":1349,"threads_started":1}
I20260809 22:49:10.546584 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.530885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48170 (local address 127.24.136.1:36179)
0809 22:49:10.532687 (+ 1802us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.532698 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.532729 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:49:10.532875 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.532885 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.536107 (+ 3222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.536297 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:10.537104 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.539424 (+ 2320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.543310 (+ 3886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.544743 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.544902 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.545140 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.546061 (+ 921us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.546241 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1602,"thread_start_us":1421,"threads_started":1}
I20260809 22:49:10.549304 15823 tablet_service.cc:1467] Processing CreateTablet for tablet 828302a940ee4ca0850fdc0a5cd5d405 (DEFAULT_TABLE table=test-table [id=f637f47b001e45728cb23dcb343c1cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:10.549345 15591 tablet_service.cc:1467] Processing CreateTablet for tablet 828302a940ee4ca0850fdc0a5cd5d405 (DEFAULT_TABLE table=test-table [id=f637f47b001e45728cb23dcb343c1cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:10.550768 15823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828302a940ee4ca0850fdc0a5cd5d405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:10.550804 15591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828302a940ee4ca0850fdc0a5cd5d405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:10.555629 15340 tablet_service.cc:1467] Processing CreateTablet for tablet 828302a940ee4ca0850fdc0a5cd5d405 (DEFAULT_TABLE table=test-table [id=f637f47b001e45728cb23dcb343c1cce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:10.557358 15340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828302a940ee4ca0850fdc0a5cd5d405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:10.574337 15989 tablet_bootstrap.cc:492] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: Bootstrap starting.
I20260809 22:49:10.576197 15992 tablet_bootstrap.cc:492] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879: Bootstrap starting.
I20260809 22:49:10.578346 15989 tablet_bootstrap.cc:654] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:10.579790 15989 log.cc:824] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:10.582072 15992 tablet_bootstrap.cc:654] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:10.582736 15995 tablet_bootstrap.cc:492] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: Bootstrap starting.
I20260809 22:49:10.585976 15995 tablet_bootstrap.cc:654] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:10.587414 15995 log.cc:824] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:10.587415 15992 log.cc:824] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:10.589668 15989 tablet_bootstrap.cc:492] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: No bootstrap required, opened a new log
I20260809 22:49:10.590003 15989 ts_tablet_manager.cc:1397] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260809 22:49:10.612113 15989 raft_consensus.cc:348] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } }
I20260809 22:49:10.612644 15989 raft_consensus.cc:374] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:10.612743 15989 raft_consensus.cc:729] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0daa246d2bc41d09bca2ccdcf2b46ff, State: Initialized, Role: FOLLOWER
I20260809 22:49:10.614307 15989 consensus_queue.cc:260] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } }
I20260809 22:49:10.616914 15992 tablet_bootstrap.cc:492] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879: No bootstrap required, opened a new log
I20260809 22:49:10.616925 15995 tablet_bootstrap.cc:492] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: No bootstrap required, opened a new log
I20260809 22:49:10.617280 15992 ts_tablet_manager.cc:1397] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879: Time spent bootstrapping tablet: real 0.041s user 0.017s sys 0.011s
I20260809 22:49:10.617429 15995 ts_tablet_manager.cc:1397] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: Time spent bootstrapping tablet: real 0.035s user 0.014s sys 0.020s
I20260809 22:49:10.628381 15939 heartbeater.cc:503] Master 127.24.136.62:38169 was elected leader, sending a full tablet report...
I20260809 22:49:10.629202 15989 ts_tablet_manager.cc:1428] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: Time spent starting tablet: real 0.039s user 0.012s sys 0.016s
I20260809 22:49:10.630056 15995 raft_consensus.cc:348] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } }
I20260809 22:49:10.630728 15995 raft_consensus.cc:374] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:10.630899 15995 raft_consensus.cc:729] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 234fb959b117472fa47182571b273420, State: Initialized, Role: FOLLOWER
I20260809 22:49:10.630913 15992 raft_consensus.cc:348] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } }
I20260809 22:49:10.631397 15992 raft_consensus.cc:374] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:10.631525 15992 raft_consensus.cc:729] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cbbebb6f1b748e8b451e0606f7c1879, State: Initialized, Role: FOLLOWER
I20260809 22:49:10.632285 15992 consensus_queue.cc:260] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } }
I20260809 22:49:10.632717 15995 consensus_queue.cc:260] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } }
I20260809 22:49:10.638302 15992 ts_tablet_manager.cc:1428] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879: Time spent starting tablet: real 0.021s user 0.018s sys 0.000s
I20260809 22:49:10.638590 15995 ts_tablet_manager.cc:1428] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: Time spent starting tablet: real 0.021s user 0.006s sys 0.013s
I20260809 22:49:10.638901 15442 heartbeater.cc:503] Master 127.24.136.62:38169 was elected leader, sending a full tablet report...
W20260809 22:49:10.666060 15940 tablet.cc:2369] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:10.701011 16009 raft_consensus.cc:482] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:49:10.701483 16009 raft_consensus.cc:504] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } }
I20260809 22:49:10.704124 16009 leader_election.cc:288] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 234fb959b117472fa47182571b273420 (127.24.136.2:33049), b0daa246d2bc41d09bca2ccdcf2b46ff (127.24.136.3:44245)
I20260809 22:49:10.711875 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.704729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:41729 (local address 127.24.136.2:33049)
0809 22:49:10.704976 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.704983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.705440 (+ 457us) server_negotiation.cc:407] Connection header received
0809 22:49:10.705745 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.705753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.705842 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.705911 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:10.706803 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.707783 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.709995 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.710859 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.711014 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.711108 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.711497 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.711632 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:49:10.712235 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.704558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:33049 (local address 127.24.136.1:41729)
0809 22:49:10.705324 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:10.705344 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:10.705501 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:10.706148 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:10.706154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:10.706185 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:10.706654 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.706666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.707922 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.707926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.709827 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.709840 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.711083 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.711089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.711309 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.711335 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:49:10.711424 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 22:49:10.711601 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":48,"thread_start_us":418,"threads_started":1}
I20260809 22:49:10.713271 15634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828302a940ee4ca0850fdc0a5cd5d405" candidate_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234fb959b117472fa47182571b273420" is_pre_election: true
I20260809 22:49:10.713903 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.705809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:48529 (local address 127.24.136.3:44245)
0809 22:49:10.705996 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.706001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.706900 (+ 899us) server_negotiation.cc:407] Connection header received
0809 22:49:10.707076 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.707081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.707155 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.707225 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:10.708059 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.709124 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.711613 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.712770 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.712923 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.712993 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.713518 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.713665 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:49:10.713847 15634 raft_consensus.cc:2454] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cbbebb6f1b748e8b451e0606f7c1879 in term 0.
I20260809 22:49:10.714015 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.705516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:44245 (local address 127.24.136.1:48529)
0809 22:49:10.706752 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:10.706770 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:10.706906 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:10.707475 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:10.707480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:10.707505 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:10.707855 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.707866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.709288 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.709293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.711399 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:10.711414 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.713143 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.713149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:10.713321 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.713354 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:49:10.713450 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:49:10.713667 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"mutex_wait_us":2,"thread_start_us":894,"threads_started":1}
I20260809 22:49:10.714795 15256 leader_election.cc:302] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cbbebb6f1b748e8b451e0606f7c1879, 234fb959b117472fa47182571b273420; no voters:
I20260809 22:49:10.715202 16009 raft_consensus.cc:2781] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:49:10.715121 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828302a940ee4ca0850fdc0a5cd5d405" candidate_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" is_pre_election: true
I20260809 22:49:10.715338 16009 raft_consensus.cc:482] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:49:10.715437 16009 raft_consensus.cc:3037] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:10.715726 15857 raft_consensus.cc:2454] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cbbebb6f1b748e8b451e0606f7c1879 in term 0.
I20260809 22:49:10.717881 16009 raft_consensus.cc:504] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } }
I20260809 22:49:10.718473 16009 leader_election.cc:288] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [CANDIDATE]: Term 1 election: Requested vote from peers 234fb959b117472fa47182571b273420 (127.24.136.2:33049), b0daa246d2bc41d09bca2ccdcf2b46ff (127.24.136.3:44245)
I20260809 22:49:10.719133 15634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828302a940ee4ca0850fdc0a5cd5d405" candidate_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234fb959b117472fa47182571b273420"
I20260809 22:49:10.719131 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828302a940ee4ca0850fdc0a5cd5d405" candidate_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff"
I20260809 22:49:10.719352 15634 raft_consensus.cc:3037] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:10.719348 15857 raft_consensus.cc:3037] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:10.721697 15634 raft_consensus.cc:2454] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cbbebb6f1b748e8b451e0606f7c1879 in term 1.
I20260809 22:49:10.722203 15256 leader_election.cc:302] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cbbebb6f1b748e8b451e0606f7c1879, 234fb959b117472fa47182571b273420; no voters:
I20260809 22:49:10.722460 15857 raft_consensus.cc:2454] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cbbebb6f1b748e8b451e0606f7c1879 in term 1.
I20260809 22:49:10.722525 16009 raft_consensus.cc:2781] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:10.723106 16009 raft_consensus.cc:686] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [term 1 LEADER]: Becoming Leader. State: Replica: 1cbbebb6f1b748e8b451e0606f7c1879, State: Running, Role: LEADER
I20260809 22:49:10.723723 16009 consensus_queue.cc:237] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } }
I20260809 22:49:10.729246 15102 catalog_manager.cc:5495] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cbbebb6f1b748e8b451e0606f7c1879 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cbbebb6f1b748e8b451e0606f7c1879" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 36179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 } health_report { overall_health: UNKNOWN } } }
W20260809 22:49:10.781098 15706 tablet.cc:2369] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:10.786813 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.780448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48174 (local address 127.24.136.1:36179)
0809 22:49:10.780852 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.780859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.780883 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:49:10.781000 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.781005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.781151 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.781321 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:10.781859 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.782657 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.784699 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.785090 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.785200 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.785731 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:10.786270 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:10.786380 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.786485 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.786556 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 22:49:10.796587 15634 raft_consensus.cc:1261] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [term 1 FOLLOWER]: Refusing update from remote peer 1cbbebb6f1b748e8b451e0606f7c1879: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:10.796608 15857 raft_consensus.cc:1261] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [term 1 FOLLOWER]: Refusing update from remote peer 1cbbebb6f1b748e8b451e0606f7c1879: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:10.797569 16009 consensus_queue.cc:1035] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [LEADER]: Connected to new peer: Peer: permanent_uuid: "234fb959b117472fa47182571b273420" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:10.797935 16020 consensus_queue.cc:1035] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0daa246d2bc41d09bca2ccdcf2b46ff" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 44245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:10.816759 16023 mvcc.cc:205] Tried to move back new op lower bound from 7316749315251343360 to 7316749314968907776. Current Snapshot: MvccSnapshot[applied={T|T < 7316749315251343360}]
I20260809 22:49:10.833061 15102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:49:10.833349 15102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:49:10.837196 15102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d980b3ee93f4626b1e06d0ecb7e875e: Sending DeleteTablet for 3 replicas of tablet 828302a940ee4ca0850fdc0a5cd5d405
I20260809 22:49:10.838734 15823 tablet_service.cc:1514] Processing DeleteTablet for tablet 828302a940ee4ca0850fdc0a5cd5d405 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:10 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:35486
I20260809 22:49:10.838913 15340 tablet_service.cc:1514] Processing DeleteTablet for tablet 828302a940ee4ca0850fdc0a5cd5d405 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:10 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:48170
I20260809 22:49:10.839679 15591 tablet_service.cc:1514] Processing DeleteTablet for tablet 828302a940ee4ca0850fdc0a5cd5d405 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:10 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56736
I20260809 22:49:10.853761 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 15174
I20260809 22:49:10.894660 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:10.854735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.24.136.1:36179)
0809 22:49:10.855437 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:10.855445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:10.855531 (+ 86us) server_negotiation.cc:407] Connection header received
0809 22:49:10.855626 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:10.855633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:10.855724 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:10.855856 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:10.858910 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.861656 (+ 2746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.863869 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:10.864631 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:10.864759 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:10.873262 (+ 8503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:10.873289 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:10.873316 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:10.873383 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:10.882529 (+ 9146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:10.886998 (+ 4469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:10.887013 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:10.887018 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:10.887291 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:10.888984 (+ 1693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:10.888991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:10.888995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:10.889623 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:10.889979 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:10.893861 (+ 3882us) server_negotiation.cc:299] Negotiation successful
0809 22:49:10.894191 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538}
I20260809 22:49:10.895527 15433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:10.901054 16043 tablet_replica.cc:330] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: stopping tablet replica
I20260809 22:49:10.901587 16043 raft_consensus.cc:2229] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:49:10.902137 16043 raft_consensus.cc:2258] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:10.903023 16038 tablet_replica.cc:330] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: stopping tablet replica
I20260809 22:49:10.903532 16038 raft_consensus.cc:2229] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:49:10.904086 16038 raft_consensus.cc:2258] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:10.906635 16043 ts_tablet_manager.cc:1905] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:10.906635 16038 ts_tablet_manager.cc:1905] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:10.914644 16038 ts_tablet_manager.cc:1918] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:10.914884 16038 log.cc:1196] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/828302a940ee4ca0850fdc0a5cd5d405
I20260809 22:49:10.915199 16043 ts_tablet_manager.cc:1918] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:10.915372 16043 log.cc:1196] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/828302a940ee4ca0850fdc0a5cd5d405
I20260809 22:49:10.915482 16038 ts_tablet_manager.cc:1939] T 828302a940ee4ca0850fdc0a5cd5d405 P b0daa246d2bc41d09bca2ccdcf2b46ff: Deleting consensus metadata
I20260809 22:49:10.915854 16043 ts_tablet_manager.cc:1939] T 828302a940ee4ca0850fdc0a5cd5d405 P 234fb959b117472fa47182571b273420: Deleting consensus metadata
I20260809 22:49:10.917477 15077 catalog_manager.cc:4841] TS 234fb959b117472fa47182571b273420 (127.24.136.2:33049): tablet 828302a940ee4ca0850fdc0a5cd5d405 (table test-table [id=f637f47b001e45728cb23dcb343c1cce]) successfully deleted
I20260809 22:49:10.917827 15077 catalog_manager.cc:4841] TS b0daa246d2bc41d09bca2ccdcf2b46ff (127.24.136.3:44245): tablet 828302a940ee4ca0850fdc0a5cd5d405 (table test-table [id=f637f47b001e45728cb23dcb343c1cce]) successfully deleted
I20260809 22:49:11.008248 16039 tablet_replica.cc:330] T 828302a940ee4ca0850fdc0a5cd5d405 P 1cbbebb6f1b748e8b451e0606f7c1879: stopping tablet replica
I20260809 22:49:11.008514 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 15174
W20260809 22:49:11.034060 15079 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.136.1:36179 (error 108)
I20260809 22:49:11.034833 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 15451
I20260809 22:49:11.036065 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.035512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36179 (local address 127.0.0.1:48184)
0809 22:49:11.035820 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:11.035926 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
W20260809 22:49:11.036345 15079 catalog_manager.cc:4584] TS 1cbbebb6f1b748e8b451e0606f7c1879 (127.24.136.1:36179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 828302a940ee4ca0850fdc0a5cd5d405: Network error: Client connection negotiation failed: client connection to 127.24.136.1:36179: connect: Connection refused (error 111)
I20260809 22:49:11.048974 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.035702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.24.136.2:33049)
0809 22:49:11.035908 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:11.035914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:11.035935 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:49:11.036122 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:11.036126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:11.036212 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:11.036335 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:11.037674 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:11.038419 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:11.039780 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:11.040197 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:11.040296 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:11.042933 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:11.042952 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:11.042969 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:11.043005 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:11.045995 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:11.046837 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:11.046846 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:11.046849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:11.047008 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:11.047421 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:11.047424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:11.047426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:11.047958 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:11.048202 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:11.048489 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 22:49:11.048670 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:49:11.049737 15695 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:11.090142 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.089629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36179 (local address 127.0.0.1:48192)
0809 22:49:11.089853 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:11.089975 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 22:49:11.091187 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.090824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36179 (local address 127.0.0.1:48200)
0809 22:49:11.090975 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:11.091075 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:49:11.141013 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 15451
I20260809 22:49:11.155110 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.154506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36179 (local address 127.0.0.1:48210)
0809 22:49:11.154756 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:11.154873 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:49:11.156356 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.155942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36179 (local address 127.0.0.1:48222)
0809 22:49:11.156112 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:11.156204 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 22:49:11.170864 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 15715
I20260809 22:49:11.185873 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.171721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35492 (local address 127.24.136.3:44245)
0809 22:49:11.171931 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:11.171937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:11.171959 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:49:11.172119 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:11.172124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:11.172197 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:11.172299 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:11.173688 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:11.174447 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:11.176336 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:11.176776 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:11.176903 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:11.179641 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:11.179660 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:11.179679 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:11.179720 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:11.182648 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:11.183556 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:11.183568 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:11.183572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:11.183730 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:11.184141 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:11.184145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:11.184147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:11.184544 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:11.184877 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:11.185301 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 22:49:11.185548 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:49:11.186619 15928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:11.236825 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.236165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36179 (local address 127.0.0.1:48226)
0809 22:49:11.236440 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:11.236573 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 22:49:11.238065 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.237549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36179 (local address 127.0.0.1:48242)
0809 22:49:11.237775 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:11.237876 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 22:49:11.279693 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 15715
I20260809 22:49:11.309372 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 15041
I20260809 22:49:11.310672 15161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:11.425060 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.424080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:36179 (local address 127.0.0.1:48246)
0809 22:49:11.424342 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:11.424431 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:36179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260809 22:49:11.425220 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 15041
I20260809 22:49:11.456140 25120 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 22:49:11.802373 25120 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 22:49:11.802613 25120 mini_ranger.cc:66] Stopping Ranger...
I20260809 22:49:12.149430 25120 mini_ranger.cc:68] Stopped Ranger
2026-08-09 22:49:12.149 UTC [13427] LOG: received smart shutdown request
2026-08-09 22:49:12.152 UTC [13427] LOG: background worker "logical replication launcher" (PID 13449) exited with exit code 1
2026-08-09 22:49:12.152 UTC [13440] LOG: shutting down
2026-08-09 22:49:12.317 UTC [13427] LOG: database system is shut down
2026-08-09T22:49:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56639 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:12 dist-test-slave-pww0 krb5kdc[16064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:12 dist-test-slave-pww0 krb5kdc[16064](info): set up 2 sockets
Aug 09 22:49:12 dist-test-slave-pww0 krb5kdc[16064](info): commencing operation
krb5kdc: starting...
W20260809 22:49:13.759498 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:49:13 dist-test-slave-pww0 krb5kdc[16064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:49:14.107734 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45081
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45081,127.24.136.61:40805,127.24.136.60:34241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:14.476758 16090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:14.477113 16090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:14.477262 16090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:14.484866 16090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:14.485006 16090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:14.485086 16090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:14.485170 16090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:14.485231 16090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:14.495923 16090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45081,127.24.136.61:40805,127.24.136.60:34241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:14.497468 16090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:14.498835 16090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:14.511636 16104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:14.511744 16105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:14.511758 16108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:14.512528 16090 server_base.cc:1034] running on GCE node
I20260809 22:49:14.513584 16090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:14.515549 16090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:14.516811 16090 hybrid_clock.cc:648] HybridClock initialized: now 1786315754516755 us; error 42 us; skew 500 ppm
I20260809 22:49:14.517287 16090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:14 dist-test-slave-pww0 krb5kdc[16064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315754, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:14.527204 16090 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:49:14.529209 16090 webserver.cc:460] Webserver started at http://127.24.136.62:45977/ using document root <none> and password file <none>
I20260809 22:49:14.530202 16090 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:14.530346 16090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:14.530678 16090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:14.534667 16090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "ce357e82efd54b0ba683b1300acdef10"
format_stamp: "Formatted at 2026-08-09 22:49:14 on dist-test-slave-pww0"
server_key: "741fce19155e7275d3e40fe4ee94daf5"
server_key_iv: "cc60e3ad36211bfde8e61d3438eb5b6c"
server_key_version: "encryptionkey@0"
I20260809 22:49:14.535755 16090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "ce357e82efd54b0ba683b1300acdef10"
format_stamp: "Formatted at 2026-08-09 22:49:14 on dist-test-slave-pww0"
server_key: "741fce19155e7275d3e40fe4ee94daf5"
server_key_iv: "cc60e3ad36211bfde8e61d3438eb5b6c"
server_key_version: "encryptionkey@0"
I20260809 22:49:14.542999 16090 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260809 22:49:14.547546 16125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:14.549254 16090 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:49:14.549443 16090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "ce357e82efd54b0ba683b1300acdef10"
format_stamp: "Formatted at 2026-08-09 22:49:14 on dist-test-slave-pww0"
server_key: "741fce19155e7275d3e40fe4ee94daf5"
server_key_iv: "cc60e3ad36211bfde8e61d3438eb5b6c"
server_key_version: "encryptionkey@0"
I20260809 22:49:14.549660 16090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:14.584496 16090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:14.589224 16090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:14.589524 16090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:14.609954 16090 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:45081
I20260809 22:49:14.609948 16213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:45081 every 8 connection(s)
I20260809 22:49:14.612001 16090 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:14.617489 16216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:14.619477 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 16090
I20260809 22:49:14.619742 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:49:14.620422 25120 external_mini_cluster.cc:1428] Setting key 5e35e4333f74585ff9ce25cec4bef0df
I20260809 22:49:14.628448 16216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.136.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.61@KRBTEST.COM" created.
Aug 09 22:49:14 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315754, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:14.660670 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.633342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57578 (local address 127.24.136.62:45081)
0809 22:49:14.635340 (+ 1998us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:14.635349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:14.635410 (+ 61us) server_negotiation.cc:407] Connection header received
0809 22:49:14.638241 (+ 2831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:14.638249 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:14.638608 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:14.638850 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:14.640840 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:14.642732 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:14.645278 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:14.646259 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:14.646507 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:14.652815 (+ 6308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:14.652825 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:14.652840 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:14.652878 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:14.656040 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:14.656820 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:14.656827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:14.656830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:14.657003 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:14.657408 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:14.657411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:14.657413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:14.657983 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:14.658171 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:14.658779 (+ 608us) server_negotiation.cc:299] Negotiation successful
0809 22:49:14.659089 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":1772,"spinlock_wait_cycles":82816,"thread_start_us":1514,"threads_started":1}
I20260809 22:49:14.660703 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.633342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45081 (local address 127.0.0.1:57578)
0809 22:49:14.634493 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:14.634556 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:14.637529 (+ 2973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:14.638940 (+ 1411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:14.638949 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:14.639966 (+ 1017us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:14.640551 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:14.640584 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:14.642932 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:14.642940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:14.645046 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:14.645089 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:14.646573 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:14.646581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:14.646883 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:14.647854 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:14.647889 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:14.652514 (+ 4625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:14.656209 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:14.656223 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:14.656237 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:14.656673 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:14.657121 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:14.657126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:14.657130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:14.657293 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:14.658127 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:14.658136 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:14.658376 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 22:49:14.658610 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 22:49:14.659225 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":203,"thread_start_us":462,"threads_started":1}
I20260809 22:49:14.664060 16216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:14.665390 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.664880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52866)
0809 22:49:14.665098 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:14.665234 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 22:49:14.665642 16127 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
I20260809 22:49:14.667261 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.666889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52868)
0809 22:49:14.667034 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:14.667114 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:49:14.667568 16216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:40805: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab.
Entry for principal kudu/127.24.136.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab.
I20260809 22:49:14.695262 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.61:40805
--webserver_interface=127.24.136.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab
--principal=kudu/127.24.136.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45081,127.24.136.61:40805,127.24.136.60:34241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:14.717063 16216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } attempt: 1
I20260809 22:49:14.718338 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.717861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52878)
0809 22:49:14.718059 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:14.718197 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 22:49:14.719316 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.718959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52880)
0809 22:49:14.719103 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:14.719180 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 22:49:14.719653 16216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:40805: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
I20260809 22:49:14.787961 16216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } attempt: 2
I20260809 22:49:14.789593 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.788884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52896)
0809 22:49:14.789216 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:14.789371 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 22:49:14.791209 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.790533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52906)
0809 22:49:14.790745 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:14.790856 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 22:49:14.791566 16216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:40805: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
I20260809 22:49:14.882905 16216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } attempt: 3
I20260809 22:49:14.884348 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.883784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52910)
0809 22:49:14.884016 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:14.884153 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 22:49:14.885651 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:14.885189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52926)
0809 22:49:14.885367 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:14.885464 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:49:14.885937 16216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:40805: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
I20260809 22:49:15.029239 16216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } attempt: 4
I20260809 22:49:15.030663 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.029991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52930)
0809 22:49:15.030381 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.030489 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260809 22:49:15.031884 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.031433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52934)
0809 22:49:15.031604 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.031722 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 22:49:15.032145 16216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:40805: Network error: Client connection negotiation failed: client connection to 127.24.136.61:40805: connect: Connection refused (error 111)
W20260809 22:49:15.063481 16226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:15.063861 16226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:15.063999 16226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:15.071662 16226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:15.071806 16226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:15.071889 16226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:15.071966 16226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:15.072029 16226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:15.082773 16226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45081,127.24.136.61:40805,127.24.136.60:34241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab
--principal=kudu/127.24.136.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.61:40805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.136.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:15.084273 16226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:15.085637 16226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:15.099876 16235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:15.100382 16238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:15.100503 16236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:15.101476 16226 server_base.cc:1034] running on GCE node
I20260809 22:49:15.102329 16226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:15.103896 16226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:15.105109 16226 hybrid_clock.cc:648] HybridClock initialized: now 1786315755105048 us; error 54 us; skew 500 ppm
I20260809 22:49:15.105470 16226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:15.108860 16226 init.cc:377] Logged in from keytab as kudu/127.24.136.61@KRBTEST.COM (short username kudu)
I20260809 22:49:15.110563 16226 webserver.cc:460] Webserver started at http://127.24.136.61:39209/ using document root <none> and password file <none>
I20260809 22:49:15.111307 16226 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:15.111436 16226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:15.111788 16226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:15.115008 16226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/data/instance:
uuid: "ad9ae093b8464159af8b051eb86ef977"
format_stamp: "Formatted at 2026-08-09 22:49:15 on dist-test-slave-pww0"
server_key: "16d589df106d7f9aefa9bdbac211dcf6"
server_key_iv: "05632a50c1ad24f16bbfa1848b558e34"
server_key_version: "encryptionkey@0"
I20260809 22:49:15.115932 16226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/wal/instance:
uuid: "ad9ae093b8464159af8b051eb86ef977"
format_stamp: "Formatted at 2026-08-09 22:49:15 on dist-test-slave-pww0"
server_key: "16d589df106d7f9aefa9bdbac211dcf6"
server_key_iv: "05632a50c1ad24f16bbfa1848b558e34"
server_key_version: "encryptionkey@0"
I20260809 22:49:15.121834 16226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:49:15.126001 16246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:15.128024 16226 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 22:49:15.128286 16226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/wal
uuid: "ad9ae093b8464159af8b051eb86ef977"
format_stamp: "Formatted at 2026-08-09 22:49:15 on dist-test-slave-pww0"
server_key: "16d589df106d7f9aefa9bdbac211dcf6"
server_key_iv: "05632a50c1ad24f16bbfa1848b558e34"
server_key_version: "encryptionkey@0"
I20260809 22:49:15.128553 16226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:15.143395 16226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:15.148108 16226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:15.148408 16226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:15.167857 16226 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.61:40805
I20260809 22:49:15.167909 16342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.61:40805 every 8 connection(s)
I20260809 22:49:15.169373 16226 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
I20260809 22:49:15.170660 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 16226
I20260809 22:49:15.170845 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-1/wal/instance
I20260809 22:49:15.171298 25120 external_mini_cluster.cc:1428] Setting key 3cffa3f53a4755b0c5839790e83bf6dc
I20260809 22:49:15.174961 16344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:15.184810 16344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.136.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.60@KRBTEST.COM" created.
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:15.219530 16348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.188428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57588 (local address 127.24.136.62:45081)
0809 22:49:15.200557 (+ 12129us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.200564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.200991 (+ 427us) server_negotiation.cc:407] Connection header received
0809 22:49:15.203349 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.203356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.203450 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.203588 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.205422 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.206156 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.207888 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.208336 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.208461 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.212283 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.212291 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.212296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.212349 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.215603 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.216690 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.216697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.216701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.216872 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.217410 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.217416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.217419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.217809 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.217957 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.218881 (+ 924us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.219114 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11989,"thread_start_us":11882,"threads_started":1}
I20260809 22:49:15.220686 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.189176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45081 (local address 127.0.0.1:57588)
0809 22:49:15.200747 (+ 11571us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.200825 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:15.202982 (+ 2157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:15.203951 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:15.203963 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:15.204545 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:15.205205 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.205225 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.206341 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.206345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.207710 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.207726 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.208598 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.208607 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.208847 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.209553 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:15.209579 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:15.212001 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:15.215792 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.215803 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.215824 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.216511 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.216992 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.216996 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.217000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.217256 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.217944 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:15.217953 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:15.218416 (+ 463us) client_negotiation.cc:769] Sending connection context
0809 22:49:15.218730 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 22:49:15.219257 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10780,"thread_start_us":10514,"threads_started":1}
I20260809 22:49:15.222730 16344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab.
Entry for principal kudu/127.24.136.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab.
I20260809 22:49:15.239532 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.60:34241
--webserver_interface=127.24.136.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab
--principal=kudu/127.24.136.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:39757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45081,127.24.136.61:40805,127.24.136.60:34241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:15.241752 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.223592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52938)
0809 22:49:15.223799 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.223817 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:15.223918 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:15.225204 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:15.225209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:15.225702 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:15.225879 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.225890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.227868 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.227874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.229190 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.229205 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.230442 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.230448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.230696 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.231398 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:15.231423 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:15.234528 (+ 3105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:15.238414 (+ 3886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.238420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.238425 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.238962 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.239525 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.239530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.239533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.239725 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.240603 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:15.240608 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:15.240770 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 22:49:15.240926 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 22:49:15.241311 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:49:15.241869 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.223904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52938 (local address 127.24.136.61:40805)
0809 22:49:15.224692 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.224699 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.224732 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:49:15.224790 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.224798 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.225031 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.225181 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.226048 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.227711 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.229352 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.230244 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.230410 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.234800 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.234819 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.234835 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.234871 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.238259 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.239164 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.239175 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.239179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.239347 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.239868 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.239872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.239875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.240433 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.240667 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.241179 (+ 512us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.241463 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":419,"threads_started":1}
I20260809 22:49:15.245360 16344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:15.247524 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.246685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50492)
0809 22:49:15.247099 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.247269 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272}
W20260809 22:49:15.248196 16254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
I20260809 22:49:15.250437 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.249955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50500)
0809 22:49:15.250145 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.250289 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 22:49:15.250861 16344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:34241: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
I20260809 22:49:15.300227 16344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } attempt: 1
I20260809 22:49:15.301698 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.301099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50506)
0809 22:49:15.301365 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.301497 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 22:49:15.302958 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.302534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50512)
0809 22:49:15.302713 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.302810 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:49:15.303273 16344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:34241: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
I20260809 22:49:15.331496 16216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } attempt: 5
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315754, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:49:15.349527 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.332337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52948)
0809 22:49:15.332650 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.332673 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:15.332834 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:15.333249 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:15.333254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:15.333822 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:15.334084 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.334100 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.335584 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.335590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.336946 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.336963 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.338041 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.338047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.338294 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.339047 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:15.339090 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:15.342043 (+ 2953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:15.346399 (+ 4356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.346405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.346409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.346971 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.347449 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.347480 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.347484 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.347684 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.348373 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:15.348378 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:15.348583 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 22:49:15.348760 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 22:49:15.349147 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260809 22:49:15.349736 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.332424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.24.136.61:40805)
0809 22:49:15.332687 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.332693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.332745 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:49:15.332965 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.332970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.333071 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.333188 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.334264 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.335424 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.337113 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.337860 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.337996 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.342391 (+ 4395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.342399 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.342404 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.342457 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.346155 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.347118 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.347131 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.347135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.347313 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.347809 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.347814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.347817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.348232 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.348393 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.348984 (+ 591us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.349369 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260809 22:49:15.351323 16216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:15.352694 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.352209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50522)
0809 22:49:15.352411 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.352514 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 22:49:15.353840 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.353422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50534)
0809 22:49:15.353585 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.353698 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:49:15.354157 16216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:34241: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
I20260809 22:49:15.371546 16344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } attempt: 2
I20260809 22:49:15.372846 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.372313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50544)
0809 22:49:15.372540 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.372655 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 22:49:15.373951 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.373573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50554)
0809 22:49:15.373723 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.373810 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:49:15.374265 16344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:34241: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
I20260809 22:49:15.405405 16216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } attempt: 1
I20260809 22:49:15.406744 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.406240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50562)
0809 22:49:15.406470 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.406568 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 22:49:15.407929 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.407559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50572)
0809 22:49:15.407730 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.407809 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 22:49:15.408172 16216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:34241: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
I20260809 22:49:15.465515 16344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } attempt: 3
I20260809 22:49:15.466753 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.466234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50582)
0809 22:49:15.466473 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.466579 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 22:49:15.468300 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.467618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50592)
0809 22:49:15.467940 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.468074 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
W20260809 22:49:15.468681 16344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:34241: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
I20260809 22:49:15.476404 16216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } attempt: 2
I20260809 22:49:15.477844 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.477205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50594)
0809 22:49:15.477496 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.477631 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 22:49:15.479046 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.478593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50600)
0809 22:49:15.478773 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.478889 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 22:49:15.479305 16216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:34241: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
I20260809 22:49:15.585579 16216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } attempt: 3
I20260809 22:49:15.586972 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.586389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50608)
0809 22:49:15.586630 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.586751 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 22:49:15.588191 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.587703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50610)
0809 22:49:15.587938 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.588039 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260809 22:49:15.588543 16216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:34241: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
I20260809 22:49:15.611945 16344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } attempt: 4
I20260809 22:49:15.613173 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.612611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50614)
0809 22:49:15.612876 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.612987 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 22:49:15.614395 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.613957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50618)
0809 22:49:15.614162 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.614257 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260809 22:49:15.614684 16344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:34241: Network error: Client connection negotiation failed: client connection to 127.24.136.60:34241: connect: Connection refused (error 111)
W20260809 22:49:15.617499 16355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:15.617868 16355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:15.618024 16355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:15.629170 16355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:15.629329 16355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:15.629390 16355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:15.629448 16355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:15.629496 16355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:15.645211 16355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:39757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45081,127.24.136.61:40805,127.24.136.60:34241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab
--principal=kudu/127.24.136.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.60:34241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.136.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:15.646979 16355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:15.648510 16355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:15.661360 16362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:15.661334 16371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:15.662045 16355 server_base.cc:1034] running on GCE node
W20260809 22:49:15.662110 16363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:15.663060 16355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:15.664996 16355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:15.666235 16355 hybrid_clock.cc:648] HybridClock initialized: now 1786315755666172 us; error 63 us; skew 500 ppm
I20260809 22:49:15.666648 16355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:15.670548 16355 init.cc:377] Logged in from keytab as kudu/127.24.136.60@KRBTEST.COM (short username kudu)
I20260809 22:49:15.672453 16355 webserver.cc:460] Webserver started at http://127.24.136.60:33755/ using document root <none> and password file <none>
I20260809 22:49:15.673357 16355 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:15.673489 16355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:15.673905 16355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:15.678167 16355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/data/instance:
uuid: "d10264df6c3243e59d7d52456b56ec7f"
format_stamp: "Formatted at 2026-08-09 22:49:15 on dist-test-slave-pww0"
server_key: "196230599840a389a03c8c07ca3737d3"
server_key_iv: "c84d38fd16cebe68dc91543d78eb051e"
server_key_version: "encryptionkey@0"
I20260809 22:49:15.679241 16355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/wal/instance:
uuid: "d10264df6c3243e59d7d52456b56ec7f"
format_stamp: "Formatted at 2026-08-09 22:49:15 on dist-test-slave-pww0"
server_key: "196230599840a389a03c8c07ca3737d3"
server_key_iv: "c84d38fd16cebe68dc91543d78eb051e"
server_key_version: "encryptionkey@0"
I20260809 22:49:15.686442 16355 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260809 22:49:15.691238 16385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:15.692953 16355 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:49:15.693145 16355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/wal
uuid: "d10264df6c3243e59d7d52456b56ec7f"
format_stamp: "Formatted at 2026-08-09 22:49:15 on dist-test-slave-pww0"
server_key: "196230599840a389a03c8c07ca3737d3"
server_key_iv: "c84d38fd16cebe68dc91543d78eb051e"
server_key_version: "encryptionkey@0"
I20260809 22:49:15.693368 16355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:15.715814 16355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:15.721318 16355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:15.721627 16355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:15.741569 16355 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.60:34241
I20260809 22:49:15.741592 16474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.60:34241 every 8 connection(s)
I20260809 22:49:15.743389 16355 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
I20260809 22:49:15.747763 16475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:15.753459 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 16355
I20260809 22:49:15.753661 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315533505708-25120-0/minicluster-data/master-2/wal/instance
I20260809 22:49:15.754257 25120 external_mini_cluster.cc:1428] Setting key 33481a73b26a89a38a16a62de01d1df9
I20260809 22:49:15.755976 16475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:15.768239 16216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } attempt: 4
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315754, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:15.790371 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.756507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50628 (local address 127.24.136.60:34241)
0809 22:49:15.759795 (+ 3288us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.759812 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.760453 (+ 641us) server_negotiation.cc:407] Connection header received
0809 22:49:15.763300 (+ 2847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.763346 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.764093 (+ 747us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.764638 (+ 545us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.767620 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.770177 (+ 2557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.772900 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.774279 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.774516 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.780550 (+ 6034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.780562 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.780580 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.780635 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.785015 (+ 4380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.786039 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.786050 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.786061 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.786300 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.786790 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.786795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.786799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.787612 (+ 813us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.787932 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.788443 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.788894 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2998,"thread_start_us":2687,"threads_started":1}
I20260809 22:49:15.795787 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.768961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50644)
0809 22:49:15.769146 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.769167 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:15.769284 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:15.777235 (+ 7951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:15.777240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:15.777742 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:15.777948 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.777963 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.779735 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.779741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.781135 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.781154 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.782505 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.782511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.782991 (+ 480us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.783742 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:15.783780 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:15.786937 (+ 3157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:15.792439 (+ 5502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.792446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.792450 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.792971 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.793495 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.793500 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.793503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.793678 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.794723 (+ 1045us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:15.794728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:15.794890 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 22:49:15.795023 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 22:49:15.795350 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 22:49:15.795809 16484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.760873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45081 (local address 127.0.0.1:57604)
0809 22:49:15.775780 (+ 14907us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.775848 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:15.776136 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:15.776754 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:15.776762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:15.777617 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:15.777960 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.777972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.779528 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.779534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.781278 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.781293 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.782636 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.782645 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.782839 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.783874 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:15.783898 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:15.787287 (+ 3389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:15.792520 (+ 5233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.792533 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.792541 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.793171 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.793622 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.793627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.793631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.793779 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.794642 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:15.794650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:15.794844 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 22:49:15.795031 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 22:49:15.795343 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14563,"thread_start_us":14371,"threads_started":1}
I20260809 22:49:15.795950 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.760139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57604 (local address 127.24.136.62:45081)
0809 22:49:15.768726 (+ 8587us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.768733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.775966 (+ 7233us) server_negotiation.cc:407] Connection header received
0809 22:49:15.776296 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.776302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.776395 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.776477 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.778165 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.779274 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.781466 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.782182 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.782323 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.787586 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.787592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.787597 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.787635 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.792348 (+ 4713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.793328 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.793334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.793338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.793474 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.793936 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.793941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.793943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.794384 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.794533 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.795184 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.795530 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":8443,"thread_start_us":8246,"threads_started":1}
I20260809 22:49:15.797479 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.769460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50644 (local address 127.24.136.60:34241)
0809 22:49:15.776839 (+ 7379us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.776846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.776870 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:49:15.776927 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.776932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.777005 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.778115 (+ 1110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.778357 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.779547 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.781648 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.782332 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.783547 (+ 1215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.787200 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.787207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.787212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.787244 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.792246 (+ 5002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.793155 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.793160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.793163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.793350 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.793829 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.793833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.794246 (+ 413us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.794590 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.796745 (+ 2155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.796871 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.797082 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7236,"thread_start_us":7060,"threads_started":1}
I20260809 22:49:15.798048 16475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:15.803771 16216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10: Bootstrap starting.
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:49:15.807829 16216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10: Neither blocks nor log segments found. Creating new log.
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:15.809165 16216 log.cc:824] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:15.815448 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.798901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52962 (local address 127.24.136.61:40805)
0809 22:49:15.799077 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.799083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.799333 (+ 250us) server_negotiation.cc:407] Connection header received
0809 22:49:15.799616 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.799621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.799702 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.799797 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.800765 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.801789 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.803512 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.804129 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.804293 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.808268 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.808275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.808280 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.808327 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.812039 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.813007 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.813013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.813017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.813208 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.813717 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.813721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.813724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.814088 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.814211 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.814794 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.815054 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 22:49:15.816241 16484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.799049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:40805 (local address 127.0.0.1:52962)
0809 22:49:15.799237 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.799256 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:15.799428 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:15.799974 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:15.799979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:15.800424 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:15.800605 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.800617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.801985 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.801990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.803310 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.803326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.804520 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.804527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.804728 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.805283 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:15.805308 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:15.807992 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:15.812241 (+ 4249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.812247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.812251 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.812839 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.813398 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.813403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.813407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.813586 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.814334 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:15.814340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:15.814510 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 22:49:15.815418 (+ 908us) client_negotiation.cc:240] Negotiation successful
0809 22:49:15.815851 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 22:49:15.817526 16475 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:15.818200 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.796930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57614 (local address 127.24.136.62:45081)
0809 22:49:15.797116 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.797121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.797138 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:49:15.797184 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.797188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.797246 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.797346 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.799432 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.800570 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.802473 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.803151 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.803277 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.809914 (+ 6637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.809922 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.809927 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.809970 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.813559 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.814608 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.814614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.814618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.814799 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.815301 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.815306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.815310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.816845 (+ 1535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.817184 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.817608 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.817868 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":62}
I20260809 22:49:15.820811 16216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10: No bootstrap required, opened a new log
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:49:15.833557 16216 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
I20260809 22:49:15.833949 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.818715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50650 (local address 127.24.136.60:34241)
0809 22:49:15.818863 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.818870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.818887 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:49:15.818938 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.818942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.819030 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.819134 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.820078 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.821068 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.822699 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.823347 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.823450 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.827570 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.827577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.827581 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.827619 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.830971 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.831836 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.831841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.831845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.831986 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.832376 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.832381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.832384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.832728 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.832864 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.833379 (+ 515us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.833627 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 22:49:15.834062 16216 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:15.834194 16216 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce357e82efd54b0ba683b1300acdef10, State: Initialized, Role: FOLLOWER
I20260809 22:49:15.834795 16484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.818486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50650)
0809 22:49:15.818677 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.818697 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:15.818799 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:15.819178 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:15.819182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:15.819705 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:15.819908 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.819922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.821227 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.821232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.822541 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.822559 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.823723 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.823729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.823900 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.824522 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:15.824547 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:15.827326 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:15.831146 (+ 3820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.831152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.831156 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.831711 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.832111 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.832115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.832119 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.832276 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.832928 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:15.832932 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:15.833088 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 22:49:15.834136 (+ 1048us) client_negotiation.cc:240] Negotiation successful
0809 22:49:15.834466 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 22:49:15.836227 16216 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
I20260809 22:49:15.840844 16475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f: Bootstrap starting.
I20260809 22:49:15.841956 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.820080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.24.136.61:40805)
0809 22:49:15.820384 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.820396 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.820415 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:49:15.820552 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.820557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.820631 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.820708 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.822474 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.823646 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.825574 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.826285 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.826381 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.831934 (+ 5553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.831942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.831946 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.831994 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.837847 (+ 5853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.839019 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.839027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.839031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.839234 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.839826 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.839833 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.839836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.840243 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.840547 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.841272 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.841572 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:49:15.845490 16475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:15.846858 16475 log.cc:824] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:15.849610 16216 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:15.850660 16475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f: No bootstrap required, opened a new log
I20260809 22:49:15.848304 16487 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:15.852463 16487 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:15.856086 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:15.862579 16475 raft_consensus.cc:348] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
I20260809 22:49:15.863080 16475 raft_consensus.cc:374] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:15.863206 16475 raft_consensus.cc:729] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d10264df6c3243e59d7d52456b56ec7f, State: Initialized, Role: FOLLOWER
I20260809 22:49:15.864107 16475 consensus_queue.cc:260] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
I20260809 22:49:15.868166 16507 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:15.868556 16507 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:15.869766 16475 sys_catalog.cc:564] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:15.909128 16487 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:49:15.909472 16487 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
I20260809 22:49:15.911227 16487 leader_election.cc:288] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad9ae093b8464159af8b051eb86ef977 (127.24.136.61:40805), d10264df6c3243e59d7d52456b56ec7f (127.24.136.60:34241)
I20260809 22:49:15.912356 16437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce357e82efd54b0ba683b1300acdef10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d10264df6c3243e59d7d52456b56ec7f" is_pre_election: true
I20260809 22:49:15.912949 16437 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce357e82efd54b0ba683b1300acdef10 in term 0.
I20260809 22:49:15.905767 16540 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:49:15.913686 16133 leader_election.cc:302] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce357e82efd54b0ba683b1300acdef10, d10264df6c3243e59d7d52456b56ec7f; no voters:
I20260809 22:49:15.913861 16344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } attempt: 5
I20260809 22:49:15.914220 16487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:49:15.914390 16487 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:49:15.914505 16487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:15.917763 16487 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
I20260809 22:49:15.919243 16437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce357e82efd54b0ba683b1300acdef10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d10264df6c3243e59d7d52456b56ec7f"
I20260809 22:49:15.919510 16437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:15.922901 16437 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce357e82efd54b0ba683b1300acdef10 in term 1.
I20260809 22:49:15.923483 16133 leader_election.cc:302] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce357e82efd54b0ba683b1300acdef10, d10264df6c3243e59d7d52456b56ec7f; no voters:
I20260809 22:49:15.928718 16487 leader_election.cc:288] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [CANDIDATE]: Term 1 election: Requested vote from peers ad9ae093b8464159af8b051eb86ef977 (127.24.136.61:40805), d10264df6c3243e59d7d52456b56ec7f (127.24.136.60:34241)
I20260809 22:49:15.928992 16542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:15.930226 16542 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [term 1 LEADER]: Becoming Leader. State: Replica: ce357e82efd54b0ba683b1300acdef10, State: Running, Role: LEADER
I20260809 22:49:15.930986 16542 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
Aug 09 22:49:15 dist-test-slave-pww0 krb5kdc[16064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
I20260809 22:49:15.909101 16355 master_runner.cc:186] Error getting master registration for 127.24.136.62:45081: OK, instance_id { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" instance_seqno: 1786315754604099 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:49:15.934512 16355 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:49:15.938449 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.895043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.24.136.61:40805)
0809 22:49:15.895259 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.895266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.900323 (+ 5057us) server_negotiation.cc:407] Connection header received
0809 22:49:15.903729 (+ 3406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.903737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.903833 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.903907 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.910330 (+ 6423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.911400 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.915477 (+ 4077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.916143 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.916252 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.926205 (+ 9953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.926213 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.926218 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.926270 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.934450 (+ 8180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.935634 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.935642 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.935646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.935827 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.936349 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.936355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.936358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.936949 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.937192 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.937836 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.938089 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 22:49:15.944133 16487 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce357e82efd54b0ba683b1300acdef10. Latest consensus state: current_term: 1 leader_uuid: "ce357e82efd54b0ba683b1300acdef10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:15.944469 16487 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:15.944860 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.897727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.24.136.60:34241)
0809 22:49:15.897939 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.897947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.908369 (+ 10422us) server_negotiation.cc:407] Connection header received
0809 22:49:15.912713 (+ 4344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.912721 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.912810 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.912884 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.917582 (+ 4698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.918415 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.920750 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.921556 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.921661 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.929215 (+ 7554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.929221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.929226 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.929267 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.937301 (+ 8034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.941572 (+ 4271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.941579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.941583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.941744 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.942251 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.942268 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.942272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.942727 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.943095 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.944239 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.944472 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:49:15.945068 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.919034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50676 (local address 127.24.136.60:34241)
0809 22:49:15.922375 (+ 3341us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.922381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.922402 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:49:15.922452 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.922456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.922535 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.922655 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.923716 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.924802 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.926685 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.927375 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.927530 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.935075 (+ 7545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.935082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.935086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.935134 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.939978 (+ 4844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.942652 (+ 2674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.942657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.942660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.942811 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.943382 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.943388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.943390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.943941 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.944074 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.944538 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.944746 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3187}
I20260809 22:49:15.945151 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.914750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:34241 (local address 127.0.0.1:50676)
0809 22:49:15.914952 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:15.914972 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:15.915082 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:15.922722 (+ 7640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:15.922728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:15.923264 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:15.923519 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.923535 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.925022 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.925028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.926450 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:15.926465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.927785 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.927792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:15.928038 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.930728 (+ 2690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:15.930753 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:15.934813 (+ 4060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:15.941933 (+ 7120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.941938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.941941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.942471 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.942963 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:15.942968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:15.942971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:15.943128 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:15.944084 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:15.944089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:15.944290 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 22:49:15.944463 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 22:49:15.944771 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 22:49:15.950466 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.892134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57618 (local address 127.24.136.62:45081)
0809 22:49:15.892374 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.892381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.899751 (+ 7370us) server_negotiation.cc:407] Connection header received
0809 22:49:15.900703 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.900712 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.900847 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.900931 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.906865 (+ 5934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.908146 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.913623 (+ 5477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:15.914336 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:15.914481 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:15.929159 (+ 14678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:15.929167 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:15.929172 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:15.929218 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:15.941716 (+ 12498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.942750 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.942782 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.942786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.942982 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:15.943832 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:15.943838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:15.943874 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:15.946625 (+ 2751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:15.948603 (+ 1978us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:15.948894 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 22:49:15.949557 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:49:15.958416 16344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977: Bootstrap starting.
I20260809 22:49:15.962652 16344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:15.964135 16344 log.cc:824] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:15.952560 16550 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:49:15.969069 16548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:15.969341 16344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977: No bootstrap required, opened a new log
W20260809 22:49:15.974184 16396 tablet.cc:2369] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:15.956894 16090 master_runner.cc:186] Error getting master registration for 127.24.136.61:40805: OK, instance_id { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" instance_seqno: 1786315755162724 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:49:15.976415 16090 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:49:15.982728 16344 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
I20260809 22:49:15.983284 16344 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:15.987429 16548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:15.983421 16344 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9ae093b8464159af8b051eb86ef977, State: Initialized, Role: FOLLOWER
I20260809 22:49:15.999600 16344 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
I20260809 22:49:16.002343 16559 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:16.005142 16285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce357e82efd54b0ba683b1300acdef10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9ae093b8464159af8b051eb86ef977"
I20260809 22:49:16.015626 16559 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:16.005095 16288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ce357e82efd54b0ba683b1300acdef10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9ae093b8464159af8b051eb86ef977" is_pre_election: true
I20260809 22:49:16.005084 16344 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:16.009874 16564 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:49:16.017997 16564 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } }
I20260809 22:49:16.019001 16288 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce357e82efd54b0ba683b1300acdef10 in term 0.
I20260809 22:49:16.022917 16172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad9ae093b8464159af8b051eb86ef977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce357e82efd54b0ba683b1300acdef10" is_pre_election: true
I20260809 22:49:16.026475 16437 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [term 1 FOLLOWER]: Refusing update from remote peer ce357e82efd54b0ba683b1300acdef10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:16.027242 16437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad9ae093b8464159af8b051eb86ef977" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d10264df6c3243e59d7d52456b56ec7f" is_pre_election: true
I20260809 22:49:16.027583 16437 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad9ae093b8464159af8b051eb86ef977 in current term 1: Already voted for candidate ce357e82efd54b0ba683b1300acdef10 in this term.
I20260809 22:49:16.027771 16487 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:16.028178 16254 leader_election.cc:302] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad9ae093b8464159af8b051eb86ef977; no voters: ce357e82efd54b0ba683b1300acdef10, d10264df6c3243e59d7d52456b56ec7f
I20260809 22:49:16.028723 16564 leader_election.cc:288] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce357e82efd54b0ba683b1300acdef10 (127.24.136.62:45081), d10264df6c3243e59d7d52456b56ec7f (127.24.136.60:34241)
I20260809 22:49:16.028831 16559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:16.032788 16559 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 22:49:16.039156 16288 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [term 1 FOLLOWER]: Refusing update from remote peer ce357e82efd54b0ba683b1300acdef10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:16.041327 16487 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:16.047305 16507 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce357e82efd54b0ba683b1300acdef10. Latest consensus state: current_term: 1 leader_uuid: "ce357e82efd54b0ba683b1300acdef10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:16.049304 16507 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:16.049654 16558 mvcc.cc:205] Tried to move back new op lower bound from 7316749336664084480 to 7316749336305303552. Current Snapshot: MvccSnapshot[applied={T|T < 7316749336664084480}]
I20260809 22:49:16.051198 16578 mvcc.cc:205] Tried to move back new op lower bound from 7316749336664084480 to 7316749336305303552. Current Snapshot: MvccSnapshot[applied={T|T < 7316749336664084480}]
I20260809 22:49:16.053459 16487 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce357e82efd54b0ba683b1300acdef10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:16.053766 16487 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:16.063328 16548 catalog_manager.cc:1347] Generated new cluster ID: 61d5f5c5f04f4adcbd5b3162233a3823
I20260809 22:49:16.063488 16548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:16.063884 16507 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce357e82efd54b0ba683b1300acdef10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:16.064177 16507 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:16.071945 16559 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce357e82efd54b0ba683b1300acdef10. Latest consensus state: current_term: 1 leader_uuid: "ce357e82efd54b0ba683b1300acdef10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:16.072309 16559 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:16.074764 16542 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce357e82efd54b0ba683b1300acdef10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:16.075109 16542 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:16.076649 16588 mvcc.cc:205] Tried to move back new op lower bound from 7316749336664084480 to 7316749336305303552. Current Snapshot: MvccSnapshot[applied={T|T < 7316749336664084480}]
I20260809 22:49:16.080348 16559 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce357e82efd54b0ba683b1300acdef10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce357e82efd54b0ba683b1300acdef10" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45081 } } peers { permanent_uuid: "ad9ae093b8464159af8b051eb86ef977" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 40805 } } peers { permanent_uuid: "d10264df6c3243e59d7d52456b56ec7f" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 34241 } } }
I20260809 22:49:16.080664 16559 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:16.080871 16595 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:49:16.091017 16548 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:16.096243 16548 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:16.117727 16548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce357e82efd54b0ba683b1300acdef10: Generated new TSK 0
I20260809 22:49:16.119042 16548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:16.919859 16540 catalog_manager.cc:1259] Loaded cluster ID: 61d5f5c5f04f4adcbd5b3162233a3823
I20260809 22:49:16.919994 16540 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f: loading cluster ID for follower catalog manager: success
I20260809 22:49:16.923995 16540 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f: acquiring CA information for follower catalog manager: success
I20260809 22:49:16.926728 16540 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d10264df6c3243e59d7d52456b56ec7f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:49:17.087196 16595 catalog_manager.cc:1259] Loaded cluster ID: 61d5f5c5f04f4adcbd5b3162233a3823
I20260809 22:49:17.087409 16595 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977: loading cluster ID for follower catalog manager: success
I20260809 22:49:17.091534 16595 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977: acquiring CA information for follower catalog manager: success
I20260809 22:49:17.094565 16595 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ad9ae093b8464159af8b051eb86ef977: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:49:17.229676 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 16090
I20260809 22:49:17.231197 16207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:17.416647 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 16090
I20260809 22:49:17.458426 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 16226
I20260809 22:49:17.459849 16336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:17.607880 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 16226
I20260809 22:49:17.646404 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 16355
I20260809 22:49:17.647732 16470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:17.787245 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 16355
2026-08-09T22:49:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5358 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:17 dist-test-slave-pww0 krb5kdc[16617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:17 dist-test-slave-pww0 krb5kdc[16617](info): set up 2 sockets
Aug 09 22:49:17 dist-test-slave-pww0 krb5kdc[16617](info): commencing operation
krb5kdc: starting...
W20260809 22:49:19.073889 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:49:19 dist-test-slave-pww0 krb5kdc[16617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:49:19.368880 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:44839
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:44597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:44839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:19.721853 16637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:19.722173 16637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:19.722312 16637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:19.729746 16637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:19.729872 16637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:19.729934 16637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:19.730023 16637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:19.730077 16637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:19.740494 16637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:44597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:44839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:44839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:19.741991 16637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:19.743402 16637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:19.755399 16647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:19.755425 16648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:19.755492 16650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:19.756471 16637 server_base.cc:1034] running on GCE node
I20260809 22:49:19.757335 16637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:19.758863 16637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:19.760061 16637 hybrid_clock.cc:648] HybridClock initialized: now 1786315759760010 us; error 43 us; skew 500 ppm
I20260809 22:49:19.760448 16637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:19 dist-test-slave-pww0 krb5kdc[16617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315759, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:19.767558 16637 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:49:19.769320 16637 webserver.cc:460] Webserver started at http://127.24.136.62:38103/ using document root <none> and password file <none>
I20260809 22:49:19.770066 16637 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:19.770182 16637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:19.770534 16637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:19.773772 16637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "3eee7da0cf9d494da913c49a4e7fd18d"
format_stamp: "Formatted at 2026-08-09 22:49:19 on dist-test-slave-pww0"
server_key: "8700843f559eb2c6e89163cc7ffb97d0"
server_key_iv: "7acf6f72b41081c85e59834b37ac8e32"
server_key_version: "encryptionkey@0"
I20260809 22:49:19.774569 16637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "3eee7da0cf9d494da913c49a4e7fd18d"
format_stamp: "Formatted at 2026-08-09 22:49:19 on dist-test-slave-pww0"
server_key: "8700843f559eb2c6e89163cc7ffb97d0"
server_key_iv: "7acf6f72b41081c85e59834b37ac8e32"
server_key_version: "encryptionkey@0"
I20260809 22:49:19.780396 16637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:49:19.784119 16664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:19.785507 16637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:49:19.785691 16637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "3eee7da0cf9d494da913c49a4e7fd18d"
format_stamp: "Formatted at 2026-08-09 22:49:19 on dist-test-slave-pww0"
server_key: "8700843f559eb2c6e89163cc7ffb97d0"
server_key_iv: "7acf6f72b41081c85e59834b37ac8e32"
server_key_version: "encryptionkey@0"
I20260809 22:49:19.785899 16637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:19.800414 16637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:19.805164 16637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:19.805477 16637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:19.824131 16637 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:44839
I20260809 22:49:19.824157 16757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:44839 every 8 connection(s)
I20260809 22:49:19.825536 16637 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:19.827667 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 16637
I20260809 22:49:19.827888 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:49:19.828559 25120 external_mini_cluster.cc:1428] Setting key ad2aae157fb498ecc2bb49e655d1bdfa
I20260809 22:49:19.829865 16758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:49:19 dist-test-slave-pww0 krb5kdc[16617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:19.844395 16758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d: Bootstrap starting.
I20260809 22:49:19.848258 16758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:19.849548 16758 log.cc:824] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:19.853047 16758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d: No bootstrap required, opened a new log
I20260809 22:49:19.853387 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:19.830428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33136 (local address 127.24.136.62:44839)
0809 22:49:19.831274 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:19.831291 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:19.831345 (+ 54us) server_negotiation.cc:407] Connection header received
0809 22:49:19.833401 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:19.833434 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:19.833999 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:19.834504 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:19.836111 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:19.838206 (+ 2095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:19.839636 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:19.840720 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:19.840917 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:19.845516 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:19.845525 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:19.845541 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:19.845589 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:19.848754 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:19.849770 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:19.849777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:19.849784 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:19.849958 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:19.850469 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:19.850474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:19.850477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:19.851186 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:19.851444 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:19.851920 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 22:49:19.852297 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":349,"threads_started":1}
I20260809 22:49:19.857404 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:19.862229 16758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eee7da0cf9d494da913c49a4e7fd18d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44839 } }
I20260809 22:49:19.862625 16758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:19.862718 16758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eee7da0cf9d494da913c49a4e7fd18d, State: Initialized, Role: FOLLOWER
I20260809 22:49:19.863237 16758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eee7da0cf9d494da913c49a4e7fd18d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44839 } }
I20260809 22:49:19.863492 16758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:19.863624 16758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:19.863770 16758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:19.865996 16758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eee7da0cf9d494da913c49a4e7fd18d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44839 } }
I20260809 22:49:19.866493 16758 leader_election.cc:302] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eee7da0cf9d494da913c49a4e7fd18d; no voters:
I20260809 22:49:19.867081 16758 leader_election.cc:288] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:19.867278 16767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:19.868021 16767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [term 1 LEADER]: Becoming Leader. State: Replica: 3eee7da0cf9d494da913c49a4e7fd18d, State: Running, Role: LEADER
I20260809 22:49:19.868636 16767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eee7da0cf9d494da913c49a4e7fd18d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44839 } }
I20260809 22:49:19.869206 16758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:19.871752 16769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3eee7da0cf9d494da913c49a4e7fd18d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eee7da0cf9d494da913c49a4e7fd18d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44839 } } }
I20260809 22:49:19.872220 16769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:19.872767 16770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eee7da0cf9d494da913c49a4e7fd18d. Latest consensus state: current_term: 1 leader_uuid: "3eee7da0cf9d494da913c49a4e7fd18d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eee7da0cf9d494da913c49a4e7fd18d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 44839 } } }
I20260809 22:49:19.873150 16770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:19.874365 16778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:19.879853 16778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:19.889292 16778 catalog_manager.cc:1347] Generated new cluster ID: f08f0b76ed4d45e8821b28d3955fc7df
I20260809 22:49:19.889425 16778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:19.905928 16778 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:19.907326 16778 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:19.916481 16778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3eee7da0cf9d494da913c49a4e7fd18d: Generated new TSK 0
I20260809 22:49:19.917393 16778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:19.971069 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 16637
I20260809 22:49:19.972352 16749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:20.069835 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 16637
2026-08-09T22:49:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2282 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:20 dist-test-slave-pww0 krb5kdc[16807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:20 dist-test-slave-pww0 krb5kdc[16807](info): set up 2 sockets
Aug 09 22:49:20 dist-test-slave-pww0 krb5kdc[16807](info): commencing operation
krb5kdc: starting...
W20260809 22:49:21.337908 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:49:21 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:49:21.627564 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:41331
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:41331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:21.987732 16834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:21.988255 16834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:21.988399 16834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:21.995913 16834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:21.996037 16834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:21.996083 16834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:21.996140 16834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:21.996183 16834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:22.006400 16834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:41331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:41331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:22.007836 16834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:22.009202 16834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:22.021270 16842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:22.021240 16849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:22.021196 16844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:22.022059 16834 server_base.cc:1034] running on GCE node
I20260809 22:49:22.022907 16834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:22.024612 16834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:22.025852 16834 hybrid_clock.cc:648] HybridClock initialized: now 1786315762025780 us; error 61 us; skew 500 ppm
I20260809 22:49:22.026226 16834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:22 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:22.029495 16834 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:49:22.031085 16834 webserver.cc:460] Webserver started at http://127.24.136.62:37289/ using document root <none> and password file <none>
I20260809 22:49:22.031836 16834 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:22.031947 16834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:22.032215 16834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:22.035351 16834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "dafdeacedc8846549a3e8eda31ef4808"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "873d0049ca377bff3b1ac20367c7a1e6"
server_key_iv: "e4c3a3267e491e26ebd9eb216aa0b41f"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.036216 16834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "dafdeacedc8846549a3e8eda31ef4808"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "873d0049ca377bff3b1ac20367c7a1e6"
server_key_iv: "e4c3a3267e491e26ebd9eb216aa0b41f"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.041785 16834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:49:22.045883 16865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:22.047390 16834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 22:49:22.047607 16834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "dafdeacedc8846549a3e8eda31ef4808"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "873d0049ca377bff3b1ac20367c7a1e6"
server_key_iv: "e4c3a3267e491e26ebd9eb216aa0b41f"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.047809 16834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:22.078378 16834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:22.082628 16834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:22.082911 16834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:22.101052 16834 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:41331
I20260809 22:49:22.101083 16947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:41331 every 8 connection(s)
I20260809 22:49:22.102464 16834 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:22.106523 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 16834
I20260809 22:49:22.106563 16948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:22.106788 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:49:22.107692 25120 external_mini_cluster.cc:1428] Setting key ad172a63e01d51d51130e8294ded8bcc
I20260809 22:49:22.120108 16948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Bootstrap starting.
Aug 09 22:49:22 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:22.123611 16948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:22.124686 16948 log.cc:824] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:22.128069 16948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: No bootstrap required, opened a new log
I20260809 22:49:22.132326 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.110032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.24.136.62:41331)
0809 22:49:22.110878 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:22.110891 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:22.110936 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:49:22.113442 (+ 2506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:22.113472 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:22.113922 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:22.114334 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:22.116098 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.117501 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.118890 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.119688 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.119868 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.124339 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:22.124348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:22.124364 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:22.124408 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:22.127871 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.128793 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.128800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.128807 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.128983 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.129414 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.129417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.129419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.129978 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:22.130233 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:22.130777 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 22:49:22.131171 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":369,"threads_started":1}
I20260809 22:49:22.136565 16948 raft_consensus.cc:348] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:22.136937 16948 raft_consensus.cc:374] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:22.137032 16948 raft_consensus.cc:729] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafdeacedc8846549a3e8eda31ef4808, State: Initialized, Role: FOLLOWER
I20260809 22:49:22.137570 16948 consensus_queue.cc:260] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:22.137810 16948 raft_consensus.cc:388] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:22.137976 16948 raft_consensus.cc:482] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:22.138134 16948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:22.140672 16948 raft_consensus.cc:504] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:22.141186 16948 leader_election.cc:302] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dafdeacedc8846549a3e8eda31ef4808; no voters:
I20260809 22:49:22.141781 16948 leader_election.cc:288] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:22.141916 16958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:22.142571 16958 raft_consensus.cc:686] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 1 LEADER]: Becoming Leader. State: Replica: dafdeacedc8846549a3e8eda31ef4808, State: Running, Role: LEADER
I20260809 22:49:22.143113 16958 consensus_queue.cc:237] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:22.143908 16948 sys_catalog.cc:564] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:22.146013 16963 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafdeacedc8846549a3e8eda31ef4808. Latest consensus state: current_term: 1 leader_uuid: "dafdeacedc8846549a3e8eda31ef4808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } } }
I20260809 22:49:22.146414 16963 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:22.147531 16959 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dafdeacedc8846549a3e8eda31ef4808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } } }
I20260809 22:49:22.147889 16959 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:22.153040 16972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:22.158125 16972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:22.168228 16972 catalog_manager.cc:1347] Generated new cluster ID: ff7cab745afb417fae825d13d0ec2a38
I20260809 22:49:22.168380 16972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:22.186354 16972 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:22.187927 16972 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:22.198151 16972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Generated new TSK 0
I20260809 22:49:22.199090 16972 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:49:22.283077 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:41331
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:22.666754 16995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:22.667071 16995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:22.667228 16995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:22.674952 16995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:22.675133 16995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:22.675297 16995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:49:22.685640 16995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:41331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:22.687008 16995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:22.688424 16995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:22.702420 17005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:22.702533 17002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:22.702538 17003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:22.703524 16995 server_base.cc:1034] running on GCE node
I20260809 22:49:22.704219 16995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:22.705255 16995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:22.706470 16995 hybrid_clock.cc:648] HybridClock initialized: now 1786315762706403 us; error 61 us; skew 500 ppm
I20260809 22:49:22.706812 16995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:22 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:22.710043 16995 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:49:22.711616 16995 webserver.cc:460] Webserver started at http://127.24.136.1:32913/ using document root <none> and password file <none>
I20260809 22:49:22.712365 16995 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:22.712491 16995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:22.712785 16995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:22.715873 16995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "05b73c39e1614c0d923a5cf51a02481c"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "5073dffc37cc1d3928d040a5ee98ff19"
server_key_iv: "3d28410d8cadc66eff1fd9077f9f72c4"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.716722 16995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "05b73c39e1614c0d923a5cf51a02481c"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "5073dffc37cc1d3928d040a5ee98ff19"
server_key_iv: "3d28410d8cadc66eff1fd9077f9f72c4"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.721987 16995 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:49:22.725770 17021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:22.727216 16995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:49:22.727398 16995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "05b73c39e1614c0d923a5cf51a02481c"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "5073dffc37cc1d3928d040a5ee98ff19"
server_key_iv: "3d28410d8cadc66eff1fd9077f9f72c4"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.727643 16995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:22.748806 16995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:22.753685 16995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:22.753979 16995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:22.755060 16995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:22.756584 16995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:22.756675 16995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:22.756824 16995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:22.756883 16995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:22.787549 16995 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:46611
I20260809 22:49:22.787592 17215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:46611 every 8 connection(s)
I20260809 22:49:22.788882 16995 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:49:22.798120 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 16995
I20260809 22:49:22.798347 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:49:22.799000 25120 external_mini_cluster.cc:1428] Setting key 7a59f5d61de6371302fa6a8fc4b2d533
Aug 09 22:49:22 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:22.808868 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.791523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:43055 (local address 127.24.136.62:41331)
0809 22:49:22.791979 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:22.791983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:22.792872 (+ 889us) server_negotiation.cc:407] Connection header received
0809 22:49:22.794739 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:22.794745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:22.794828 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:22.794929 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:22.796576 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.797320 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.798954 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.799389 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.799529 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.803043 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:22.803048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:22.803052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:22.803085 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:22.805543 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.806320 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.806325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.806327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.806462 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.806913 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.806917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.806919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.807194 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:22.807337 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:22.808144 (+ 807us) server_negotiation.cc:299] Negotiation successful
0809 22:49:22.808499 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":239,"threads_started":1}
I20260809 22:49:22.809909 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.791990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:43055)
0809 22:49:22.792705 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.792749 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:22.794454 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:22.795195 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:22.795204 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:22.795791 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:22.796384 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.796401 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.797495 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.797499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.798796 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.798808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.799650 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.799658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.799880 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.800585 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:22.800614 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:22.802809 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:22.805695 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.805705 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.805724 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.806179 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.806572 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.806578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.806582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.806792 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.807332 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:22.807340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:22.807741 (+ 401us) client_negotiation.cc:769] Sending connection context
0809 22:49:22.808004 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 22:49:22.808730 (+ 726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":260,"threads_started":1}
I20260809 22:49:22.811681 17216 heartbeater.cc:347] Connected to a master server at 127.24.136.62:41331
I20260809 22:49:22.812016 17216 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:22.812945 17216 heartbeater.cc:511] Master 127.24.136.62:41331 requested a full tablet report, sending...
I20260809 22:49:22.815234 16891 ts_manager.cc:194] Registered new tserver with Master: 05b73c39e1614c0d923a5cf51a02481c (127.24.136.1:46611)
I20260809 22:49:22.816761 16891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:43055
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:49:22.861065 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:41331
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:23.220016 17224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:23.220325 17224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:23.220431 17224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:23.228012 17224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:23.228133 17224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:23.228256 17224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:49:23.238662 17224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:41331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:23.240119 17224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:23.241480 17224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:23.254823 17233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:23.254869 17232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:23.255061 17236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:23.255836 17224 server_base.cc:1034] running on GCE node
I20260809 22:49:23.256471 17224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:23.257464 17224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:23.258670 17224 hybrid_clock.cc:648] HybridClock initialized: now 1786315763258605 us; error 59 us; skew 500 ppm
I20260809 22:49:23.258987 17224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:23 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315763, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:23.262087 17224 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:49:23.263603 17224 webserver.cc:460] Webserver started at http://127.24.136.2:36501/ using document root <none> and password file <none>
I20260809 22:49:23.264307 17224 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:23.264428 17224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:23.264722 17224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:23.268010 17224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "6275fa4eddb4407ab5b70fe4d795846b"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "7d455d2e5d77225e7d36c71f79fa8036"
server_key_iv: "573feddb4e5721b0a5f666d0112a8572"
server_key_version: "encryptionkey@0"
I20260809 22:49:23.268894 17224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "6275fa4eddb4407ab5b70fe4d795846b"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "7d455d2e5d77225e7d36c71f79fa8036"
server_key_iv: "573feddb4e5721b0a5f666d0112a8572"
server_key_version: "encryptionkey@0"
I20260809 22:49:23.274331 17224 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:49:23.278165 17246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:23.279666 17224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:49:23.279848 17224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "6275fa4eddb4407ab5b70fe4d795846b"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "7d455d2e5d77225e7d36c71f79fa8036"
server_key_iv: "573feddb4e5721b0a5f666d0112a8572"
server_key_version: "encryptionkey@0"
I20260809 22:49:23.280056 17224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:23.305692 17224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:23.310259 17224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:23.310582 17224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:23.311635 17224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:23.313059 17224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:23.313141 17224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:23.313267 17224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:23.313320 17224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:23.345170 17224 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:33379
I20260809 22:49:23.345244 17459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:33379 every 8 connection(s)
I20260809 22:49:23.346531 17224 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:49:23.350804 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 17224
I20260809 22:49:23.350987 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:49:23.351541 25120 external_mini_cluster.cc:1428] Setting key 576f7704775d0874571ced3553d0aa1c
Aug 09 22:49:23 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315763, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:23.367264 17462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.349113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:40975 (local address 127.24.136.62:41331)
0809 22:49:23.349535 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.349540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.350566 (+ 1026us) server_negotiation.cc:407] Connection header received
0809 22:49:23.352811 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.352817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.352894 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.352999 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.354793 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.355533 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.357111 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.357553 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.357667 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.361221 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.361227 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.361230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.361268 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.363922 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.364771 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.364776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.364779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.364920 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.365377 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.365380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.365382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.365668 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.365851 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.366616 (+ 765us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.366881 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":206,"threads_started":1}
I20260809 22:49:23.368360 17464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.349606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:40975)
0809 22:49:23.350398 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.350437 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.352393 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.353282 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.353292 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.353935 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:23.354599 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.354623 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.355688 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.355692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.356958 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.356971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.357786 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.357795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.358036 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.358686 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:23.358710 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:23.360959 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:23.364085 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.364096 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.364116 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.364629 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.365036 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.365042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.365046 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.365253 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.365802 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:23.365811 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:23.366209 (+ 398us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.366493 (+ 284us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.367085 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":290,"threads_started":1}
I20260809 22:49:23.369930 17460 heartbeater.cc:347] Connected to a master server at 127.24.136.62:41331
I20260809 22:49:23.370321 17460 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:23.371331 17460 heartbeater.cc:511] Master 127.24.136.62:41331 requested a full tablet report, sending...
I20260809 22:49:23.373137 16891 ts_manager.cc:194] Registered new tserver with Master: 6275fa4eddb4407ab5b70fe4d795846b (127.24.136.2:33379)
I20260809 22:49:23.374217 16891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:40975
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:49:23.414830 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:41331
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:23.755071 17468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:23.755402 17468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:23.755574 17468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:23.762876 17468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:23.763015 17468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:23.763173 17468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:49:23.773229 17468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:41331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:23.774616 17468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:23.776014 17468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:23.789330 17483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:23.789469 17486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:23.789836 17488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:23.790165 17468 server_base.cc:1034] running on GCE node
I20260809 22:49:23.791103 17468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:23.792236 17468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:23.793444 17468 hybrid_clock.cc:648] HybridClock initialized: now 1786315763793386 us; error 51 us; skew 500 ppm
I20260809 22:49:23.793751 17468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:23 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315763, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:23.796902 17468 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:49:23.798327 17468 webserver.cc:460] Webserver started at http://127.24.136.3:39989/ using document root <none> and password file <none>
I20260809 22:49:23.798985 17468 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:23.799088 17468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:23.799350 17468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:23.802393 17468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "17e4de2999c646c69f96a9a24b6ebf4a"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "90308d53c8857277821c76ec04383189"
server_key_iv: "796abb7af2ea80539698b2472c278d66"
server_key_version: "encryptionkey@0"
I20260809 22:49:23.803145 17468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "17e4de2999c646c69f96a9a24b6ebf4a"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "90308d53c8857277821c76ec04383189"
server_key_iv: "796abb7af2ea80539698b2472c278d66"
server_key_version: "encryptionkey@0"
I20260809 22:49:23.808553 17468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:49:23.812153 17497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:23.813536 17468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:49:23.813690 17468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "17e4de2999c646c69f96a9a24b6ebf4a"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "90308d53c8857277821c76ec04383189"
server_key_iv: "796abb7af2ea80539698b2472c278d66"
server_key_version: "encryptionkey@0"
I20260809 22:49:23.813884 17468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:23.820127 17216 heartbeater.cc:503] Master 127.24.136.62:41331 was elected leader, sending a full tablet report...
I20260809 22:49:23.828526 17468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:23.832934 17468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:23.833238 17468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:23.834259 17468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:23.835691 17468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:23.835768 17468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:23.835867 17468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:23.835923 17468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:23.866973 17468 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:32899
I20260809 22:49:23.867051 17695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:32899 every 8 connection(s)
I20260809 22:49:23.868463 17468 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:49:23.873061 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 17468
I20260809 22:49:23.873226 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:49:23.873662 25120 external_mini_cluster.cc:1428] Setting key ba1aa779e2af585da8365cc62e121ba3
Aug 09 22:49:23 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315763, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:23.888623 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.871059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:59757 (local address 127.24.136.62:41331)
0809 22:49:23.871565 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.871571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.872453 (+ 882us) server_negotiation.cc:407] Connection header received
0809 22:49:23.874319 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.874325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.874425 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.874530 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.876347 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.877263 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.878841 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.879265 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.879364 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.882793 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.882798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.882802 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.882839 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.885446 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.886174 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.886178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.886181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.886307 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.886712 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.886716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.886718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.887003 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.887146 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.888008 (+ 862us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.888257 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":215,"threads_started":1}
I20260809 22:49:23.889707 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.871571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:59757)
0809 22:49:23.872298 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.872338 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.874045 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.874790 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.874798 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.875421 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:23.876176 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.876198 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.877413 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.877417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.878687 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.878700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.879485 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.879493 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.879705 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.880347 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:23.880369 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:23.882566 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:23.885587 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.885594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.885608 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.886062 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.886386 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.886389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.886391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.886576 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.887100 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:23.887106 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:23.887588 (+ 482us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.887869 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.888390 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":264,"threads_started":1}
I20260809 22:49:23.890982 17696 heartbeater.cc:347] Connected to a master server at 127.24.136.62:41331
I20260809 22:49:23.891345 17696 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:23.892140 17696 heartbeater.cc:511] Master 127.24.136.62:41331 requested a full tablet report, sending...
I20260809 22:49:23.893559 16891 ts_manager.cc:194] Registered new tserver with Master: 17e4de2999c646c69f96a9a24b6ebf4a (127.24.136.3:32899)
I20260809 22:49:23.894367 16891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:59757
I20260809 22:49:23.902279 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:49:23.919648 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.907029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.24.136.62:41331)
0809 22:49:23.907209 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.907214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.907563 (+ 349us) server_negotiation.cc:407] Connection header received
0809 22:49:23.907738 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.907742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.907829 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.907919 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.909347 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.910092 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.911550 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.911974 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.912069 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.914543 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.914548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.914551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.914589 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.917104 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.917811 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.917815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.917817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.917946 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.918300 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.918304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.918306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.918661 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.918845 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.919168 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.919365 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:49:23.923821 16891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:49:23.930725 16891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:49:23.954523 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.943954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:33379 (local address 127.0.0.1:33420)
0809 22:49:23.945719 (+ 1765us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.945736 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.945856 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.946614 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.946622 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.946666 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:23.947133 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.947147 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.949109 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.949114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.951677 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.951692 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.953478 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.953484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.953672 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.953725 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.953880 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.954085 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1646,"mutex_wait_us":2,"thread_start_us":542,"threads_started":1}
I20260809 22:49:23.955582 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.945833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:32899 (local address 127.0.0.1:56046)
0809 22:49:23.947710 (+ 1877us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.947724 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.947840 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.948411 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.948416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.948439 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:23.948913 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.948924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.950322 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.950327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.952509 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.952525 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.954682 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.954688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.954876 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.954908 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.954993 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.955129 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1781,"mutex_wait_us":4,"thread_start_us":1646,"threads_started":1}
I20260809 22:49:23.954448 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.944176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.24.136.2:33379)
0809 22:49:23.945776 (+ 1600us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.945785 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.945819 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:49:23.946085 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.946094 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.946367 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.946585 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:23.947382 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.948937 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.951847 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.953276 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.953419 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.953642 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.953940 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.954122 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1445,"thread_start_us":1356,"threads_started":1}
I20260809 22:49:23.955730 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.946332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.24.136.3:32899)
0809 22:49:23.947253 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.947262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.947832 (+ 570us) server_negotiation.cc:407] Connection header received
0809 22:49:23.948046 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.948052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.948230 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.948407 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:23.949087 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.950132 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.952690 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.954176 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.954318 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.954543 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.955191 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.955366 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":595,"threads_started":1}
I20260809 22:49:23.955935 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.943236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:46611 (local address 127.0.0.1:43508)
0809 22:49:23.944961 (+ 1725us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.945016 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.945243 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.948485 (+ 3242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.948495 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.948522 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:23.949019 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.949032 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.951201 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.951206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.953181 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.953197 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.955128 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.955135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.955297 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.955328 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.955401 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.955560 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1428,"thread_start_us":2230,"threads_started":1}
I20260809 22:49:23.956782 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.943603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.24.136.1:46611)
0809 22:49:23.947890 (+ 4287us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.947898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.947926 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:49:23.948019 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.948027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.948257 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.949204 (+ 947us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:23.949465 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.951026 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.953361 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.954893 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.956211 (+ 1318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.956378 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.956436 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.956551 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4095,"thread_start_us":2653,"threads_started":1}
I20260809 22:49:23.959803 17337 tablet_service.cc:1467] Processing CreateTablet for tablet 06b7345bffd1410cbd59ab2379ee264e (DEFAULT_TABLE table=test-table [id=490208d0762b4a199c1f129a73ae2c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:23.960667 17576 tablet_service.cc:1467] Processing CreateTablet for tablet 06b7345bffd1410cbd59ab2379ee264e (DEFAULT_TABLE table=test-table [id=490208d0762b4a199c1f129a73ae2c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:23.961479 17337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b7345bffd1410cbd59ab2379ee264e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:23.962137 17576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b7345bffd1410cbd59ab2379ee264e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:23.962000 17108 tablet_service.cc:1467] Processing CreateTablet for tablet 06b7345bffd1410cbd59ab2379ee264e (DEFAULT_TABLE table=test-table [id=490208d0762b4a199c1f129a73ae2c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:23.963143 17108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06b7345bffd1410cbd59ab2379ee264e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:23.975536 17728 tablet_bootstrap.cc:492] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: Bootstrap starting.
I20260809 22:49:23.977944 17731 tablet_bootstrap.cc:492] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: Bootstrap starting.
I20260809 22:49:23.978870 17730 tablet_bootstrap.cc:492] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: Bootstrap starting.
I20260809 22:49:23.979733 17728 tablet_bootstrap.cc:654] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:23.981251 17728 log.cc:824] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:23.982512 17731 tablet_bootstrap.cc:654] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:23.982642 17730 tablet_bootstrap.cc:654] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:23.984153 17730 log.cc:824] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:23.984859 17731 log.cc:824] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:23.985154 17728 tablet_bootstrap.cc:492] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: No bootstrap required, opened a new log
I20260809 22:49:23.985456 17728 ts_tablet_manager.cc:1397] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260809 22:49:23.987757 17730 tablet_bootstrap.cc:492] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: No bootstrap required, opened a new log
I20260809 22:49:23.988077 17730 ts_tablet_manager.cc:1397] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 22:49:23.989758 17731 tablet_bootstrap.cc:492] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: No bootstrap required, opened a new log
I20260809 22:49:23.990031 17731 ts_tablet_manager.cc:1397] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260809 22:49:23.996462 17728 raft_consensus.cc:348] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:23.996989 17728 raft_consensus.cc:374] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:23.997145 17728 raft_consensus.cc:729] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b73c39e1614c0d923a5cf51a02481c, State: Initialized, Role: FOLLOWER
I20260809 22:49:23.997870 17728 consensus_queue.cc:260] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:23.998044 17731 raft_consensus.cc:348] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:23.998546 17731 raft_consensus.cc:374] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:23.998696 17731 raft_consensus.cc:729] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e4de2999c646c69f96a9a24b6ebf4a, State: Initialized, Role: FOLLOWER
I20260809 22:49:23.998484 17730 raft_consensus.cc:348] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:23.999006 17730 raft_consensus.cc:374] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:23.999141 17730 raft_consensus.cc:729] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6275fa4eddb4407ab5b70fe4d795846b, State: Initialized, Role: FOLLOWER
I20260809 22:49:23.999521 17731 consensus_queue.cc:260] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:23.999955 17728 ts_tablet_manager.cc:1428] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260809 22:49:23.999905 17730 consensus_queue.cc:260] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:24.001044 17696 heartbeater.cc:503] Master 127.24.136.62:41331 was elected leader, sending a full tablet report...
I20260809 22:49:24.001644 17731 ts_tablet_manager.cc:1428] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: Time spent starting tablet: real 0.011s user 0.003s sys 0.009s
I20260809 22:49:24.002583 17460 heartbeater.cc:503] Master 127.24.136.62:41331 was elected leader, sending a full tablet report...
I20260809 22:49:24.003247 17730 ts_tablet_manager.cc:1428] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260809 22:49:24.041378 17219 tablet.cc:2369] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:49:24.098662 17461 tablet.cc:2369] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:49:24.120098 17699 tablet.cc:2369] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:24.322505 17736 raft_consensus.cc:482] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:49:24.322844 17736 raft_consensus.cc:504] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:24.324657 17736 leader_election.cc:288] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b73c39e1614c0d923a5cf51a02481c (127.24.136.1:46611), 6275fa4eddb4407ab5b70fe4d795846b (127.24.136.2:33379)
I20260809 22:49:24.330315 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.324939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:46611 (local address 127.24.136.3:33363)
0809 22:49:24.325245 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:24.325268 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:24.325409 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:24.325859 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:24.325864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:24.325893 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:24.326085 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:24.326094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.327115 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.327119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:24.328484 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:24.328497 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.329587 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.329593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:24.329770 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.329797 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 22:49:24.329878 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 22:49:24.330016 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260809 22:49:24.330322 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.325131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:33363 (local address 127.24.136.1:46611)
0809 22:49:24.325373 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.325380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.325404 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:49:24.325647 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.325653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.325736 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.325820 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:24.326232 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.326978 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.328653 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.329436 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.329587 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.329679 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.329920 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.330049 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:49:24.331076 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.325492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:32979 (local address 127.24.136.2:33379)
0809 22:49:24.325702 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.325708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.326089 (+ 381us) server_negotiation.cc:407] Connection header received
0809 22:49:24.326279 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.326283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.326348 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.326426 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:24.326993 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.327732 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.329402 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.330171 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.330323 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.330395 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.330773 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.330879 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:49:24.331284 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.325154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:33379 (local address 127.24.136.3:32979)
0809 22:49:24.325779 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:24.325794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:24.326142 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:24.326465 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:24.326470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:24.326496 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:24.326820 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:24.326830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.327885 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.327890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:24.329238 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:24.329252 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.330404 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.330410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:24.330595 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.330627 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:49:24.330725 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 22:49:24.330860 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":5,"thread_start_us":381,"threads_started":1}
I20260809 22:49:24.331682 17134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b7345bffd1410cbd59ab2379ee264e" candidate_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b73c39e1614c0d923a5cf51a02481c" is_pre_election: true
I20260809 22:49:24.332273 17134 raft_consensus.cc:2454] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e4de2999c646c69f96a9a24b6ebf4a in term 0.
I20260809 22:49:24.332465 17375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b7345bffd1410cbd59ab2379ee264e" candidate_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6275fa4eddb4407ab5b70fe4d795846b" is_pre_election: true
I20260809 22:49:24.333022 17375 raft_consensus.cc:2454] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e4de2999c646c69f96a9a24b6ebf4a in term 0.
I20260809 22:49:24.333065 17502 leader_election.cc:302] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b73c39e1614c0d923a5cf51a02481c, 17e4de2999c646c69f96a9a24b6ebf4a; no voters:
I20260809 22:49:24.333526 17736 raft_consensus.cc:2781] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:49:24.333690 17736 raft_consensus.cc:482] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:49:24.333801 17736 raft_consensus.cc:3037] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:24.336143 17736 raft_consensus.cc:504] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:24.336805 17736 leader_election.cc:288] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [CANDIDATE]: Term 1 election: Requested vote from peers 05b73c39e1614c0d923a5cf51a02481c (127.24.136.1:46611), 6275fa4eddb4407ab5b70fe4d795846b (127.24.136.2:33379)
I20260809 22:49:24.337421 17375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b7345bffd1410cbd59ab2379ee264e" candidate_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6275fa4eddb4407ab5b70fe4d795846b"
I20260809 22:49:24.337421 17134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06b7345bffd1410cbd59ab2379ee264e" candidate_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b73c39e1614c0d923a5cf51a02481c"
I20260809 22:49:24.337661 17134 raft_consensus.cc:3037] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:24.337661 17375 raft_consensus.cc:3037] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:24.341490 17134 raft_consensus.cc:2454] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e4de2999c646c69f96a9a24b6ebf4a in term 1.
I20260809 22:49:24.341492 17375 raft_consensus.cc:2454] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e4de2999c646c69f96a9a24b6ebf4a in term 1.
I20260809 22:49:24.342193 17505 leader_election.cc:302] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 05b73c39e1614c0d923a5cf51a02481c, 17e4de2999c646c69f96a9a24b6ebf4a, 6275fa4eddb4407ab5b70fe4d795846b; no voters:
I20260809 22:49:24.342628 17736 raft_consensus.cc:2781] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:24.343119 17736 raft_consensus.cc:686] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 LEADER]: Becoming Leader. State: Replica: 17e4de2999c646c69f96a9a24b6ebf4a, State: Running, Role: LEADER
I20260809 22:49:24.343715 17736 consensus_queue.cc:237] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:24.349020 16890 catalog_manager.cc:5495] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e4de2999c646c69f96a9a24b6ebf4a (127.24.136.3). New cstate: current_term: 1 leader_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } health_report { overall_health: UNKNOWN } } }
I20260809 22:49:24.413789 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.408114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.24.136.3:32899)
0809 22:49:24.408357 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.408364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.408462 (+ 98us) server_negotiation.cc:407] Connection header received
0809 22:49:24.408723 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.408727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.408812 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.408949 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.409344 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.410092 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.411578 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.411989 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.412116 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.412667 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.413190 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.413332 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.413430 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.413517 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 22:49:24.422488 17134 raft_consensus.cc:1261] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [term 1 FOLLOWER]: Refusing update from remote peer 17e4de2999c646c69f96a9a24b6ebf4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:24.422488 17375 raft_consensus.cc:1261] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [term 1 FOLLOWER]: Refusing update from remote peer 17e4de2999c646c69f96a9a24b6ebf4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:24.423527 17740 consensus_queue.cc:1035] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:24.423879 17736 consensus_queue.cc:1035] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:24.455873 16891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:49:24.456266 16891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:49:24.462007 16891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Sending DeleteTablet for 3 replicas of tablet 06b7345bffd1410cbd59ab2379ee264e
I20260809 22:49:24.463575 17337 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b7345bffd1410cbd59ab2379ee264e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:24 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:33420
I20260809 22:49:24.463697 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 16995
I20260809 22:49:24.463852 17108 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b7345bffd1410cbd59ab2379ee264e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:24 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:43508
I20260809 22:49:24.464579 17576 tablet_service.cc:1514] Processing DeleteTablet for tablet 06b7345bffd1410cbd59ab2379ee264e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:24 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56046
I20260809 22:49:24.464764 17760 tablet_replica.cc:330] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: stopping tablet replica
I20260809 22:49:24.465270 17760 raft_consensus.cc:2229] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:49:24.465891 17760 raft_consensus.cc:2258] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:24.468020 17764 tablet_replica.cc:330] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: stopping tablet replica
I20260809 22:49:24.468394 17764 raft_consensus.cc:2229] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:49:24.468966 17764 raft_consensus.cc:2258] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:24.469699 17759 tablet_replica.cc:330] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: stopping tablet replica
I20260809 22:49:24.470130 17759 raft_consensus.cc:2229] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:49:24.470715 17759 raft_consensus.cc:2258] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:24.471192 17764 ts_tablet_manager.cc:1905] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:24.471156 17760 ts_tablet_manager.cc:1905] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:24.472610 17759 ts_tablet_manager.cc:1905] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:24.478243 17764 ts_tablet_manager.cc:1918] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:24.478498 17764 log.cc:1196] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/06b7345bffd1410cbd59ab2379ee264e
I20260809 22:49:24.478523 17760 ts_tablet_manager.cc:1918] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:24.479130 17764 ts_tablet_manager.cc:1939] T 06b7345bffd1410cbd59ab2379ee264e P 17e4de2999c646c69f96a9a24b6ebf4a: Deleting consensus metadata
I20260809 22:49:24.479147 17760 log.cc:1196] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/06b7345bffd1410cbd59ab2379ee264e
I20260809 22:49:24.479264 17759 ts_tablet_manager.cc:1918] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:24.479446 17759 log.cc:1196] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/06b7345bffd1410cbd59ab2379ee264e
I20260809 22:49:24.479768 17760 ts_tablet_manager.cc:1939] T 06b7345bffd1410cbd59ab2379ee264e P 05b73c39e1614c0d923a5cf51a02481c: Deleting consensus metadata
I20260809 22:49:24.480026 17759 ts_tablet_manager.cc:1939] T 06b7345bffd1410cbd59ab2379ee264e P 6275fa4eddb4407ab5b70fe4d795846b: Deleting consensus metadata
I20260809 22:49:24.481504 16869 catalog_manager.cc:4841] TS 17e4de2999c646c69f96a9a24b6ebf4a (127.24.136.3:32899): tablet 06b7345bffd1410cbd59ab2379ee264e (table test-table [id=490208d0762b4a199c1f129a73ae2c33]) successfully deleted
I20260809 22:49:24.481761 16872 catalog_manager.cc:4841] TS 6275fa4eddb4407ab5b70fe4d795846b (127.24.136.2:33379): tablet 06b7345bffd1410cbd59ab2379ee264e (table test-table [id=490208d0762b4a199c1f129a73ae2c33]) successfully deleted
I20260809 22:49:24.482020 16869 catalog_manager.cc:4841] TS 05b73c39e1614c0d923a5cf51a02481c (127.24.136.1:46611): tablet 06b7345bffd1410cbd59ab2379ee264e (table test-table [id=490208d0762b4a199c1f129a73ae2c33]) successfully deleted
Aug 09 22:49:24 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:49:24.500976 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.465221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.24.136.1:46611)
0809 22:49:24.465449 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.465456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.484019 (+ 18563us) server_negotiation.cc:407] Connection header received
0809 22:49:24.484248 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.484262 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.484364 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.484476 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.485981 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.487105 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.488575 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.489208 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.489356 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.493623 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:24.493642 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:24.493663 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:24.493712 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:24.497561 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:24.498361 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:24.498372 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:24.498376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:24.498554 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:24.498985 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:24.498991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:24.498994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:24.499589 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:24.499937 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.500311 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.500574 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 22:49:24.501735 17207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:24.591974 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 16995
I20260809 22:49:24.627712 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 17224
Aug 09 22:49:24 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:49:24.651096 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.628538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33424 (local address 127.24.136.2:33379)
0809 22:49:24.628739 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.628745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.628769 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:49:24.628890 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.628896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.628982 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.629108 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.630581 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.631767 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.633263 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.633911 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.634046 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.643538 (+ 9492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:24.643562 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:24.643583 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:24.643642 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:24.647746 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:24.648654 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:24.648665 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:24.648669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:24.648850 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:24.649338 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:24.649343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:24.649347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:24.649842 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:24.650162 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.650445 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.650691 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:49:24.651891 17452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:24.772269 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 17224
I20260809 22:49:24.801929 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 17468
Aug 09 22:49:24 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:49:24.822754 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.802910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.24.136.3:32899)
0809 22:49:24.803120 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.803126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.803146 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:49:24.803215 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.803219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.803297 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.803388 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.804757 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.805704 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.807069 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.807765 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.807894 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.816261 (+ 8367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:24.816282 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:24.816304 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:24.816354 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:24.819686 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:24.820468 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:24.820477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:24.820479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:24.820668 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:24.821096 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:24.821100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:24.821102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:24.821487 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:24.821845 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.822123 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.822354 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:49:24.823690 17689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:24.949757 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 17468
I20260809 22:49:24.980942 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 16834
I20260809 22:49:24.982149 16939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:25.093403 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 16834
I20260809 22:49:25.124358 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:41331
--webserver_interface=127.24.136.62
--webserver_port=37289
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:41331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:25.480930 17778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:25.481287 17778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:25.481418 17778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:25.488989 17778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:25.489138 17778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:25.489229 17778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:25.489313 17778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:25.489377 17778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:25.499902 17778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:41331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:41331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=37289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:25.501302 17778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:25.502691 17778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:25.531881 17788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:25.531931 17793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:25.531872 17789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:25.533010 17778 server_base.cc:1034] running on GCE node
I20260809 22:49:25.533946 17778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:25.535714 17778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:25.536959 17778 hybrid_clock.cc:648] HybridClock initialized: now 1786315765536899 us; error 41 us; skew 500 ppm
I20260809 22:49:25.537386 17778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:25 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315765, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:25.541503 17778 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:49:25.543412 17778 webserver.cc:460] Webserver started at http://127.24.136.62:37289/ using document root <none> and password file <none>
I20260809 22:49:25.544289 17778 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:25.544435 17778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:25.551323 17778 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 22:49:25.554886 17809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:25.556538 17778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:49:25.556735 17778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "dafdeacedc8846549a3e8eda31ef4808"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "873d0049ca377bff3b1ac20367c7a1e6"
server_key_iv: "e4c3a3267e491e26ebd9eb216aa0b41f"
server_key_version: "encryptionkey@0"
I20260809 22:49:25.557898 17778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:25.571341 17778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:25.575762 17778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:25.576037 17778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:25.597201 17778 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:41331
I20260809 22:49:25.597223 17895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:41331 every 8 connection(s)
I20260809 22:49:25.598827 17778 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:25.605261 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 17778
I20260809 22:49:25.606138 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:46611
--local_ip_for_outbound_sockets=127.24.136.1
--tserver_master_addrs=127.24.136.62:41331
--webserver_port=32913
--webserver_interface=127.24.136.1
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:25.606670 17898 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:49:25.610250 17898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Bootstrap starting.
I20260809 22:49:25.635371 17898 log.cc:824] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:25.652617 17898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:49:25.653306 17898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Bootstrap complete.
I20260809 22:49:25.667081 17898 raft_consensus.cc:348] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:25.668468 17898 raft_consensus.cc:729] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafdeacedc8846549a3e8eda31ef4808, State: Initialized, Role: FOLLOWER
I20260809 22:49:25.669296 17898 consensus_queue.cc:260] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:25.669601 17898 raft_consensus.cc:388] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:25.669747 17898 raft_consensus.cc:482] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:25.669958 17898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:49:25.676043 17898 raft_consensus.cc:504] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:25.676750 17898 leader_election.cc:302] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dafdeacedc8846549a3e8eda31ef4808; no voters:
I20260809 22:49:25.677444 17898 leader_election.cc:288] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:49:25.677623 17908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:49:25.678807 17908 raft_consensus.cc:686] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 2 LEADER]: Becoming Leader. State: Replica: dafdeacedc8846549a3e8eda31ef4808, State: Running, Role: LEADER
I20260809 22:49:25.679333 17908 consensus_queue.cc:237] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:25.679842 17898 sys_catalog.cc:564] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:25.681650 17911 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafdeacedc8846549a3e8eda31ef4808. Latest consensus state: current_term: 2 leader_uuid: "dafdeacedc8846549a3e8eda31ef4808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } } }
I20260809 22:49:25.682046 17911 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:25.682624 17910 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dafdeacedc8846549a3e8eda31ef4808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } } }
I20260809 22:49:25.682915 17910 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:25.685159 17915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:25.694103 17915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:25.695778 17915 catalog_manager.cc:1259] Loaded cluster ID: ff7cab745afb417fae825d13d0ec2a38
I20260809 22:49:25.695928 17915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:25.699424 17915 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 22:49:25.717136 17915 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 22:49:25.730964 17915 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:25.733139 17915 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Loaded TSK: 0
I20260809 22:49:25.735158 17915 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:49:26.012836 17905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:26.013159 17905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:26.013296 17905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:26.020960 17905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:26.021101 17905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:26.021247 17905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:49:26.031523 17905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:46611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=32913
--tserver_master_addrs=127.24.136.62:41331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:26.032945 17905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:26.034363 17905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:26.050094 17942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:26.050113 17943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:26.051012 17947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:26.051117 17905 server_base.cc:1034] running on GCE node
I20260809 22:49:26.051841 17905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:26.052889 17905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:26.054136 17905 hybrid_clock.cc:648] HybridClock initialized: now 1786315766054067 us; error 49 us; skew 500 ppm
I20260809 22:49:26.054556 17905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:26 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:26.058998 17905 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:49:26.060907 17905 webserver.cc:460] Webserver started at http://127.24.136.1:32913/ using document root <none> and password file <none>
I20260809 22:49:26.061919 17905 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:26.062091 17905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:26.069710 17905 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:49:26.073371 17957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:26.075179 17905 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 22:49:26.075389 17905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "05b73c39e1614c0d923a5cf51a02481c"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "5073dffc37cc1d3928d040a5ee98ff19"
server_key_iv: "3d28410d8cadc66eff1fd9077f9f72c4"
server_key_version: "encryptionkey@0"
I20260809 22:49:26.076792 17905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:26.100085 17905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:26.104501 17905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:26.104782 17905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:26.105835 17905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:26.107264 17905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:26.107362 17905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:26.107486 17905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:26.107569 17905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 22:49:26.139700 17905 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:46611
I20260809 22:49:26.139786 18140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:46611 every 8 connection(s)
I20260809 22:49:26.141250 17905 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:49:26.142724 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 17905
I20260809 22:49:26.143597 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:33379
--local_ip_for_outbound_sockets=127.24.136.2
--tserver_master_addrs=127.24.136.62:41331
--webserver_port=36501
--webserver_interface=127.24.136.2
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:49:26 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:26.169734 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.144972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:44477)
0809 22:49:26.145912 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:26.145959 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:26.148165 (+ 2206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:26.150091 (+ 1926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:26.150101 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:26.150745 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:26.151324 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:26.151343 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.153685 (+ 2342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.153690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:26.155018 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:26.155035 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.156277 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.156286 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:26.156574 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:26.157279 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:26.157310 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:26.159951 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:26.164815 (+ 4864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:26.164825 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:26.164840 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:26.165436 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:26.166100 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:26.166105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:26.166108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:26.166370 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:26.167507 (+ 1137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:26.167516 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:26.167900 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 22:49:26.168181 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 22:49:26.168641 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":387,"threads_started":1}
I20260809 22:49:26.170506 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.144755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:44477 (local address 127.24.136.62:41331)
0809 22:49:26.145469 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:26.145488 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:26.146118 (+ 630us) server_negotiation.cc:407] Connection header received
0809 22:49:26.148870 (+ 2752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:26.148889 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:26.149443 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:26.149905 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:26.151618 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.153428 (+ 1810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.155221 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.155980 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.156204 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:26.160357 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:26.160370 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:26.160393 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:26.160465 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:26.164570 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:26.165627 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:26.165637 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:26.165646 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:26.165907 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:26.166506 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:26.166511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:26.166514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:26.167268 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:26.167534 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:26.168553 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0809 22:49:26.169118 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":298,"threads_started":1}
I20260809 22:49:26.173949 18141 heartbeater.cc:347] Connected to a master server at 127.24.136.62:41331
I20260809 22:49:26.174351 18141 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:26.175407 18141 heartbeater.cc:511] Master 127.24.136.62:41331 requested a full tablet report, sending...
I20260809 22:49:26.178346 17831 ts_manager.cc:194] Registered new tserver with Master: 05b73c39e1614c0d923a5cf51a02481c (127.24.136.1:46611)
I20260809 22:49:26.180639 17831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:44477
W20260809 22:49:26.498555 18146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:26.498880 18146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:26.498976 18146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:26.506536 18146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:26.506676 18146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:26.506793 18146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:49:26.516891 18146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:33379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=36501
--tserver_master_addrs=127.24.136.62:41331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:26.518213 18146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:26.519547 18146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:26.533720 18160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:26.533844 18161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:26.534061 18164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:26.534828 18146 server_base.cc:1034] running on GCE node
I20260809 22:49:26.535568 18146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:26.536592 18146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:26.537850 18146 hybrid_clock.cc:648] HybridClock initialized: now 1786315766537805 us; error 58 us; skew 500 ppm
I20260809 22:49:26.538187 18146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:26 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:26.541867 18146 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:49:26.543587 18146 webserver.cc:460] Webserver started at http://127.24.136.2:36501/ using document root <none> and password file <none>
I20260809 22:49:26.544304 18146 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:26.544433 18146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:26.550257 18146 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 22:49:26.553298 18178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:26.554903 18146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:49:26.555069 18146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "6275fa4eddb4407ab5b70fe4d795846b"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "7d455d2e5d77225e7d36c71f79fa8036"
server_key_iv: "573feddb4e5721b0a5f666d0112a8572"
server_key_version: "encryptionkey@0"
I20260809 22:49:26.556257 18146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:26.567600 18146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:26.571854 18146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:26.572124 18146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:26.573148 18146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:26.574555 18146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:26.574649 18146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:26.574730 18146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:26.574784 18146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:26.607002 18146 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:33379
I20260809 22:49:26.607060 18348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:33379 every 8 connection(s)
I20260809 22:49:26.608922 18146 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:49:26.616776 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 18146
I20260809 22:49:26.617502 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:32899
--local_ip_for_outbound_sockets=127.24.136.3
--tserver_master_addrs=127.24.136.62:41331
--webserver_port=39989
--webserver_interface=127.24.136.3
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:49:26 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:26.631804 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.611736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:37571 (local address 127.24.136.62:41331)
0809 22:49:26.611975 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:26.611981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:26.613261 (+ 1280us) server_negotiation.cc:407] Connection header received
0809 22:49:26.615158 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:26.615165 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:26.615270 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:26.615384 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:26.617270 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.618073 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.620398 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.620855 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.620971 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:26.625521 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:26.625527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:26.625530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:26.625569 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:26.628203 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:26.628994 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:26.628998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:26.629000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:26.629139 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:26.629802 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:26.629810 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:26.629814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:26.630189 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:26.630325 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:26.631289 (+ 964us) server_negotiation.cc:299] Negotiation successful
0809 22:49:26.631537 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:49:26.633175 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.612216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:37571)
0809 22:49:26.613066 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:26.613114 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:26.614869 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:26.615650 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:26.615658 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:26.616195 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:26.617070 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:26.617098 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.618260 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.618266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:26.620181 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:26.620198 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.621116 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.621123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:26.621426 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:26.622278 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:26.622311 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:26.625245 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:26.628362 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:26.628373 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:26.628393 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:26.628851 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:26.629336 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:26.629341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:26.629345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:26.629620 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:26.630338 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:26.630348 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:26.630812 (+ 464us) client_negotiation.cc:769] Sending connection context
0809 22:49:26.631132 (+ 320us) client_negotiation.cc:240] Negotiation successful
0809 22:49:26.631722 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":346,"threads_started":1}
I20260809 22:49:26.634872 18351 heartbeater.cc:347] Connected to a master server at 127.24.136.62:41331
I20260809 22:49:26.635267 18351 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:26.636157 18351 heartbeater.cc:511] Master 127.24.136.62:41331 requested a full tablet report, sending...
I20260809 22:49:26.637861 17831 ts_manager.cc:194] Registered new tserver with Master: 6275fa4eddb4407ab5b70fe4d795846b (127.24.136.2:33379)
I20260809 22:49:26.638728 17831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:37571
W20260809 22:49:26.967902 18360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:26.968225 18360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:26.968356 18360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:26.975756 18360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:26.975904 18360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:26.976054 18360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:49:26.987063 18360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:32899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=39989
--tserver_master_addrs=127.24.136.62:41331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:26.988622 18360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:26.990141 18360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:27.004052 18365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:27.004093 18369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:27.004104 18366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:27.004680 18360 server_base.cc:1034] running on GCE node
I20260809 22:49:27.005463 18360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:27.006520 18360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:27.007766 18360 hybrid_clock.cc:648] HybridClock initialized: now 1786315767007705 us; error 39 us; skew 500 ppm
I20260809 22:49:27.008152 18360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:27 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:27.011828 18360 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:49:27.013463 18360 webserver.cc:460] Webserver started at http://127.24.136.3:39989/ using document root <none> and password file <none>
I20260809 22:49:27.014204 18360 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:27.014333 18360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:27.020342 18360 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:49:27.023660 18382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:27.025385 18360 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:49:27.025561 18360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "17e4de2999c646c69f96a9a24b6ebf4a"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "90308d53c8857277821c76ec04383189"
server_key_iv: "796abb7af2ea80539698b2472c278d66"
server_key_version: "encryptionkey@0"
I20260809 22:49:27.026620 18360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:27.045835 18360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:27.050416 18360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:27.050704 18360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:27.051826 18360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:27.053341 18360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:27.053447 18360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:27.053565 18360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:27.053637 18360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:27.085525 18360 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:32899
I20260809 22:49:27.085582 18571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:32899 every 8 connection(s)
I20260809 22:49:27.087123 18360 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:49:27.090816 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 18360
Aug 09 22:49:27 dist-test-slave-pww0 krb5kdc[16807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:27.110401 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.091930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55148 (local address 127.24.136.62:41331)
0809 22:49:27.095141 (+ 3211us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.095145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.095161 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:49:27.095205 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.095208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.095261 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.095343 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.097502 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.098338 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.100242 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.100711 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.100825 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.103892 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.103900 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.103905 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.103947 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.107179 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.108119 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.108123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.108126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.108287 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.108767 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.108772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.108775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.109208 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.109484 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.109788 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.109975 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3108,"thread_start_us":3008,"threads_started":1}
I20260809 22:49:27.112447 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.090183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:52723 (local address 127.24.136.62:41331)
0809 22:49:27.090426 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.090432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.091830 (+ 1398us) server_negotiation.cc:407] Connection header received
0809 22:49:27.094747 (+ 2917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.094751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.094834 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.094945 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.097393 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.098216 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.100574 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.101165 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.101298 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.105399 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.105405 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.105408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.105458 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.108683 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.109722 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.109728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.109731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.109892 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.110431 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.110435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.110437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.110814 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.110954 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.111962 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.112184 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:49:27.113600 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.090757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:52723)
0809 22:49:27.091653 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.091699 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.094357 (+ 2658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.095285 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.095296 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.096167 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:27.097069 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.097100 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.098406 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.098411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.100419 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.100433 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.101418 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.101424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.101770 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.102384 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:27.102409 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:27.105074 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:27.108853 (+ 3779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.108864 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.108885 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.109569 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.110018 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.110024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.110027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.110314 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.110946 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:27.110955 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:27.111417 (+ 462us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.111783 (+ 366us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.112334 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":337,"threads_started":1}
I20260809 22:49:27.115147 18572 heartbeater.cc:347] Connected to a master server at 127.24.136.62:41331
I20260809 22:49:27.115521 18572 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:27.116364 18572 heartbeater.cc:511] Master 127.24.136.62:41331 requested a full tablet report, sending...
I20260809 22:49:27.117978 17831 ts_manager.cc:194] Registered new tserver with Master: 17e4de2999c646c69f96a9a24b6ebf4a (127.24.136.3:32899)
I20260809 22:49:27.118839 17831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:52723
I20260809 22:49:27.126442 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:49:27.134732 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.127602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.24.136.62:41331)
0809 22:49:27.127837 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.127843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.128936 (+ 1093us) server_negotiation.cc:407] Connection header received
0809 22:49:27.129053 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.129058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.129274 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.129433 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:27.129972 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.130865 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.132578 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.133017 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.133131 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.133655 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:27.134159 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:27.134325 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.134390 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.134479 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:49:27.135823 17831 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:49:27.143430 17831 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:49:27.181367 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.157642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.24.136.2:33379)
0809 22:49:27.167498 (+ 9856us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.167510 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.170739 (+ 3229us) server_negotiation.cc:407] Connection header received
0809 22:49:27.171688 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.171698 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.172041 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.172292 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:27.173169 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.174681 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.177939 (+ 3258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.179618 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.179771 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.179991 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.180783 (+ 792us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.180985 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9586,"thread_start_us":9413,"threads_started":1}
I20260809 22:49:27.181393 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.157011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:33379 (local address 127.0.0.1:33438)
0809 22:49:27.170539 (+ 13528us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.170565 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.171394 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.172364 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.172378 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.172412 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:27.172913 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.172930 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.175328 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.175335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.177725 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.177741 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.180145 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.180150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.180352 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.180498 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.180710 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.180950 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13336,"thread_start_us":16095,"threads_started":1}
I20260809 22:49:27.181926 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.158045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.24.136.1:46611)
0809 22:49:27.171037 (+ 12992us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.171048 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.173068 (+ 2020us) server_negotiation.cc:407] Connection header received
0809 22:49:27.173333 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.173343 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.173555 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.173751 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:27.174892 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.176490 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.178852 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.180349 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.180485 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.180719 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.181342 (+ 623us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.181513 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12795,"thread_start_us":12996,"threads_started":1}
I20260809 22:49:27.182018 18588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.156344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:32899 (local address 127.0.0.1:56066)
0809 22:49:27.157172 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.157217 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.157404 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.173729 (+ 16325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.173735 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.173765 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:27.174210 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.174225 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.176581 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.176593 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.178860 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.178879 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.181238 (+ 2359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.181245 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.181410 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.181442 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.181503 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.181630 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":30,"thread_start_us":414,"threads_started":1}
I20260809 22:49:27.182889 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.157585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:46611 (local address 127.0.0.1:43536)
0809 22:49:27.172874 (+ 15289us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.172904 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.173075 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.174094 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.174100 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.174136 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:27.174613 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.174629 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.176704 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.176710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.178684 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.178698 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.180893 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.180901 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.181088 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.181139 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.182345 (+ 1206us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.182482 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15128,"mutex_wait_us":62,"thread_start_us":13219,"threads_started":1}
I20260809 22:49:27.183192 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.156700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.24.136.3:32899)
0809 22:49:27.173086 (+ 16386us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.173095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.173127 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:49:27.173223 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.173230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.173497 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.173706 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:27.174533 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.176365 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.179064 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.180978 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.182356 (+ 1378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.182599 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.182672 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.182850 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16183,"thread_start_us":15998,"threads_started":1}
I20260809 22:49:27.186020 18251 tablet_service.cc:1467] Processing CreateTablet for tablet 91af315e176d4200b71bd26c16e41332 (DEFAULT_TABLE table=test-table [id=839685cd9e7d429da30b0a4d135882b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:27.187362 18251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91af315e176d4200b71bd26c16e41332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:27.186884 18451 tablet_service.cc:1467] Processing CreateTablet for tablet 91af315e176d4200b71bd26c16e41332 (DEFAULT_TABLE table=test-table [id=839685cd9e7d429da30b0a4d135882b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:27.186888 18039 tablet_service.cc:1467] Processing CreateTablet for tablet 91af315e176d4200b71bd26c16e41332 (DEFAULT_TABLE table=test-table [id=839685cd9e7d429da30b0a4d135882b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:27.188450 18039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91af315e176d4200b71bd26c16e41332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:27.188786 18451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91af315e176d4200b71bd26c16e41332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:27.193352 18141 heartbeater.cc:503] Master 127.24.136.62:41331 was elected leader, sending a full tablet report...
I20260809 22:49:27.203176 18599 tablet_bootstrap.cc:492] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: Bootstrap starting.
I20260809 22:49:27.204730 18600 tablet_bootstrap.cc:492] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: Bootstrap starting.
I20260809 22:49:27.206307 18601 tablet_bootstrap.cc:492] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: Bootstrap starting.
I20260809 22:49:27.208810 18600 tablet_bootstrap.cc:654] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:27.209290 18599 tablet_bootstrap.cc:654] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:27.209561 18601 tablet_bootstrap.cc:654] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:27.210211 18600 log.cc:824] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:27.210556 18601 log.cc:824] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:27.210546 18599 log.cc:824] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:27.213593 18601 tablet_bootstrap.cc:492] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: No bootstrap required, opened a new log
I20260809 22:49:27.213716 18599 tablet_bootstrap.cc:492] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: No bootstrap required, opened a new log
I20260809 22:49:27.213711 18600 tablet_bootstrap.cc:492] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: No bootstrap required, opened a new log
I20260809 22:49:27.213852 18601 ts_tablet_manager.cc:1397] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 22:49:27.213976 18599 ts_tablet_manager.cc:1397] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20260809 22:49:27.213976 18600 ts_tablet_manager.cc:1397] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 22:49:27.221521 18601 raft_consensus.cc:348] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:27.221943 18601 raft_consensus.cc:374] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:27.222045 18601 raft_consensus.cc:729] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e4de2999c646c69f96a9a24b6ebf4a, State: Initialized, Role: FOLLOWER
I20260809 22:49:27.222687 18601 consensus_queue.cc:260] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:27.223896 18572 heartbeater.cc:503] Master 127.24.136.62:41331 was elected leader, sending a full tablet report...
I20260809 22:49:27.224665 18601 ts_tablet_manager.cc:1428] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260809 22:49:27.224696 18600 raft_consensus.cc:348] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:27.224699 18599 raft_consensus.cc:348] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:27.225190 18600 raft_consensus.cc:374] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:27.225191 18599 raft_consensus.cc:374] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:27.225301 18599 raft_consensus.cc:729] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6275fa4eddb4407ab5b70fe4d795846b, State: Initialized, Role: FOLLOWER
I20260809 22:49:27.225301 18600 raft_consensus.cc:729] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b73c39e1614c0d923a5cf51a02481c, State: Initialized, Role: FOLLOWER
I20260809 22:49:27.226004 18599 consensus_queue.cc:260] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:27.226004 18600 consensus_queue.cc:260] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:27.230392 18351 heartbeater.cc:503] Master 127.24.136.62:41331 was elected leader, sending a full tablet report...
I20260809 22:49:27.230815 18600 ts_tablet_manager.cc:1428] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: Time spent starting tablet: real 0.017s user 0.008s sys 0.010s
I20260809 22:49:27.231001 18599 ts_tablet_manager.cc:1428] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: Time spent starting tablet: real 0.017s user 0.014s sys 0.002s
W20260809 22:49:27.338966 18573 tablet.cc:2369] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:49:27.361068 18352 tablet.cc:2369] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:49:27.393795 18142 tablet.cc:2369] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:27.483000 18607 raft_consensus.cc:482] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:49:27.483371 18607 raft_consensus.cc:504] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:27.485281 18607 leader_election.cc:288] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b73c39e1614c0d923a5cf51a02481c (127.24.136.1:46611), 6275fa4eddb4407ab5b70fe4d795846b (127.24.136.2:33379)
I20260809 22:49:27.491581 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.485808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53593 (local address 127.24.136.2:33379)
0809 22:49:27.486052 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.486058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.486083 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:49:27.486337 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.486343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.486433 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.486541 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:27.486959 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.488000 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.489803 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.490642 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.490797 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.490877 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.491160 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.491294 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:49:27.491639 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.485663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:33379 (local address 127.24.136.3:53593)
0809 22:49:27.485984 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.486006 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.486156 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.486572 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.486576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.486605 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:27.486808 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.486821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.488132 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.488138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.489637 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.489650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.490797 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.490803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.490992 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.491018 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.491111 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.491262 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":1}
I20260809 22:49:27.492225 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.485737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:54921 (local address 127.24.136.1:46611)
0809 22:49:27.486088 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.486094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.486736 (+ 642us) server_negotiation.cc:407] Connection header received
0809 22:49:27.486896 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.486901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.486979 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.487047 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:27.487727 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.488472 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.490294 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.491075 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.491232 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.491326 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.491879 (+ 553us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.492025 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260809 22:49:27.492842 18613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.485831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:46611 (local address 127.24.136.3:54921)
0809 22:49:27.486613 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.486633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.486767 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.487103 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.487107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.487132 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:27.487526 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.487539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.488660 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.488665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.490101 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.490114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.491289 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.491295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.491652 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.491691 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.492420 (+ 729us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.492550 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":24,"thread_start_us":433,"threads_started":1}
I20260809 22:49:27.492682 18283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91af315e176d4200b71bd26c16e41332" candidate_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6275fa4eddb4407ab5b70fe4d795846b" is_pre_election: true
I20260809 22:49:27.493218 18283 raft_consensus.cc:2454] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e4de2999c646c69f96a9a24b6ebf4a in term 0.
I20260809 22:49:27.493978 18389 leader_election.cc:302] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e4de2999c646c69f96a9a24b6ebf4a, 6275fa4eddb4407ab5b70fe4d795846b; no voters:
I20260809 22:49:27.493939 18066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91af315e176d4200b71bd26c16e41332" candidate_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b73c39e1614c0d923a5cf51a02481c" is_pre_election: true
I20260809 22:49:27.494451 18607 raft_consensus.cc:2781] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:49:27.494592 18607 raft_consensus.cc:482] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:49:27.494524 18066 raft_consensus.cc:2454] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e4de2999c646c69f96a9a24b6ebf4a in term 0.
I20260809 22:49:27.494711 18607 raft_consensus.cc:3037] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:27.497097 18607 raft_consensus.cc:504] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:27.497749 18607 leader_election.cc:288] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [CANDIDATE]: Term 1 election: Requested vote from peers 05b73c39e1614c0d923a5cf51a02481c (127.24.136.1:46611), 6275fa4eddb4407ab5b70fe4d795846b (127.24.136.2:33379)
I20260809 22:49:27.498445 18066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91af315e176d4200b71bd26c16e41332" candidate_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b73c39e1614c0d923a5cf51a02481c"
I20260809 22:49:27.498459 18283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91af315e176d4200b71bd26c16e41332" candidate_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6275fa4eddb4407ab5b70fe4d795846b"
I20260809 22:49:27.498710 18283 raft_consensus.cc:3037] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:27.498693 18066 raft_consensus.cc:3037] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:27.502010 18283 raft_consensus.cc:2454] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e4de2999c646c69f96a9a24b6ebf4a in term 1.
I20260809 22:49:27.502010 18066 raft_consensus.cc:2454] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e4de2999c646c69f96a9a24b6ebf4a in term 1.
I20260809 22:49:27.502568 18389 leader_election.cc:302] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e4de2999c646c69f96a9a24b6ebf4a, 6275fa4eddb4407ab5b70fe4d795846b; no voters:
I20260809 22:49:27.502952 18607 raft_consensus.cc:2781] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:27.503435 18607 raft_consensus.cc:686] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 LEADER]: Becoming Leader. State: Replica: 17e4de2999c646c69f96a9a24b6ebf4a, State: Running, Role: LEADER
I20260809 22:49:27.504026 18607 consensus_queue.cc:237] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } }
I20260809 22:49:27.509218 17831 catalog_manager.cc:5495] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e4de2999c646c69f96a9a24b6ebf4a (127.24.136.3). New cstate: current_term: 1 leader_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17e4de2999c646c69f96a9a24b6ebf4a" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 32899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 } health_report { overall_health: UNKNOWN } } }
I20260809 22:49:27.559717 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.553612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56070 (local address 127.24.136.3:32899)
0809 22:49:27.553814 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.553819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.553898 (+ 79us) server_negotiation.cc:407] Connection header received
0809 22:49:27.554104 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.554108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.554196 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.554334 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:27.554879 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.555685 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.557316 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.557761 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.557860 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.558431 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:27.559036 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:27.559207 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.559296 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.559380 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:49:27.568813 18066 raft_consensus.cc:1261] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [term 1 FOLLOWER]: Refusing update from remote peer 17e4de2999c646c69f96a9a24b6ebf4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:27.568822 18283 raft_consensus.cc:1261] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [term 1 FOLLOWER]: Refusing update from remote peer 17e4de2999c646c69f96a9a24b6ebf4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:27.569937 18607 consensus_queue.cc:1035] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b73c39e1614c0d923a5cf51a02481c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:27.570290 18615 consensus_queue.cc:1035] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6275fa4eddb4407ab5b70fe4d795846b" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:27.583422 18618 mvcc.cc:205] Tried to move back new op lower bound from 7316749383950106624 to 7316749383700647936. Current Snapshot: MvccSnapshot[applied={T|T < 7316749383950106624}]
I20260809 22:49:27.600116 17831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:49:27.600380 17831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:49:27.604174 17831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Sending DeleteTablet for 3 replicas of tablet 91af315e176d4200b71bd26c16e41332
I20260809 22:49:27.605590 18451 tablet_service.cc:1514] Processing DeleteTablet for tablet 91af315e176d4200b71bd26c16e41332 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:27 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56066
I20260809 22:49:27.605849 18039 tablet_service.cc:1514] Processing DeleteTablet for tablet 91af315e176d4200b71bd26c16e41332 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:27 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:43536
I20260809 22:49:27.606431 18251 tablet_service.cc:1514] Processing DeleteTablet for tablet 91af315e176d4200b71bd26c16e41332 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:27 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:33438
I20260809 22:49:27.611961 18635 tablet_replica.cc:330] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: stopping tablet replica
I20260809 22:49:27.612515 18635 raft_consensus.cc:2229] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:49:27.613267 18635 raft_consensus.cc:2258] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:27.616286 18636 tablet_replica.cc:330] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: stopping tablet replica
I20260809 22:49:27.616760 18636 raft_consensus.cc:2229] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:49:27.617234 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 17905
I20260809 22:49:27.617283 18636 raft_consensus.cc:2258] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:27.619177 18635 ts_tablet_manager.cc:1905] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:27.631121 18637 tablet_replica.cc:330] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: stopping tablet replica
I20260809 22:49:27.631840 18637 raft_consensus.cc:2229] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:49:27.632472 18637 raft_consensus.cc:2258] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:27.633646 18636 ts_tablet_manager.cc:1905] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:27.635152 18637 ts_tablet_manager.cc:1905] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:27.637341 18635 ts_tablet_manager.cc:1918] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:27.637553 18635 log.cc:1196] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/91af315e176d4200b71bd26c16e41332
I20260809 22:49:27.638211 18635 ts_tablet_manager.cc:1939] T 91af315e176d4200b71bd26c16e41332 P 17e4de2999c646c69f96a9a24b6ebf4a: Deleting consensus metadata
I20260809 22:49:27.640291 17811 catalog_manager.cc:4841] TS 17e4de2999c646c69f96a9a24b6ebf4a (127.24.136.3:32899): tablet 91af315e176d4200b71bd26c16e41332 (table test-table [id=839685cd9e7d429da30b0a4d135882b6]) successfully deleted
I20260809 22:49:27.640671 18636 ts_tablet_manager.cc:1918] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:27.640872 18636 log.cc:1196] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/91af315e176d4200b71bd26c16e41332
I20260809 22:49:27.641480 18636 ts_tablet_manager.cc:1939] T 91af315e176d4200b71bd26c16e41332 P 05b73c39e1614c0d923a5cf51a02481c: Deleting consensus metadata
I20260809 22:49:27.640446 18637 ts_tablet_manager.cc:1918] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:27.643505 18637 log.cc:1196] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/91af315e176d4200b71bd26c16e41332
I20260809 22:49:27.643563 17811 catalog_manager.cc:4841] TS 05b73c39e1614c0d923a5cf51a02481c (127.24.136.1:46611): tablet 91af315e176d4200b71bd26c16e41332 (table test-table [id=839685cd9e7d429da30b0a4d135882b6]) successfully deleted
I20260809 22:49:27.644095 18637 ts_tablet_manager.cc:1939] T 91af315e176d4200b71bd26c16e41332 P 6275fa4eddb4407ab5b70fe4d795846b: Deleting consensus metadata
I20260809 22:49:27.646358 17814 catalog_manager.cc:4841] TS 6275fa4eddb4407ab5b70fe4d795846b (127.24.136.2:33379): tablet 91af315e176d4200b71bd26c16e41332 (table test-table [id=839685cd9e7d429da30b0a4d135882b6]) successfully deleted
I20260809 22:49:27.656987 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.618451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43546 (local address 127.24.136.1:46611)
0809 22:49:27.618611 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.618617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.633356 (+ 14739us) server_negotiation.cc:407] Connection header received
0809 22:49:27.633544 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.633549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.633672 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.633777 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.642690 (+ 8913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.643646 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.645757 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.646238 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.646368 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.650032 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.650050 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.650070 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.650118 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.653805 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.654854 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.654863 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.654865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.655016 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.655493 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.655498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.655501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.655880 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.656137 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.656535 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.656718 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 22:49:27.657727 18135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:27.760810 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 17905
I20260809 22:49:27.800582 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 18146
I20260809 22:49:27.817057 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.801418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33440 (local address 127.24.136.2:33379)
0809 22:49:27.801598 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.801603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.801663 (+ 60us) server_negotiation.cc:407] Connection header received
0809 22:49:27.801845 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.801849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.801920 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.802003 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.803977 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.804797 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.806808 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.807220 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.807325 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.810801 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.810819 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.810836 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.810881 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.813965 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.814880 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.814887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.814890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.815041 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.815582 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.815586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.815589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.816002 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.816226 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.816587 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.816783 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:49:27.817751 18340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:27.907807 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 18146
I20260809 22:49:27.944655 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 18360
I20260809 22:49:27.962337 18589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.945528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.24.136.3:32899)
0809 22:49:27.945736 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.945743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.946023 (+ 280us) server_negotiation.cc:407] Connection header received
0809 22:49:27.946242 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.946246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.946315 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.946422 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.948314 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.949197 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.951246 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.951843 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.951954 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.955729 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.955747 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.955763 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.955820 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.958910 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.960017 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.960026 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.960028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.960226 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.960689 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.960693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.960695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.961085 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.961397 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.961744 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.961951 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:49:27.963156 18563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:28.089679 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 18360
I20260809 22:49:28.115432 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 17778
I20260809 22:49:28.116719 17887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:28.231082 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 17778
I20260809 22:49:28.266149 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:41331
--webserver_interface=127.24.136.62
--webserver_port=37289
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:41331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:28.633842 18645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:28.634169 18645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:28.634325 18645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:28.641762 18645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:28.641907 18645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:28.641991 18645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:28.642069 18645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:28.642130 18645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:28.652374 18645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:41331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:41331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=37289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:28.653832 18645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:28.655180 18645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:28.666960 18656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:28.666960 18657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:28.667121 18659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:28.667976 18645 server_base.cc:1034] running on GCE node
I20260809 22:49:28.668817 18645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:28.670353 18645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:28.671568 18645 hybrid_clock.cc:648] HybridClock initialized: now 1786315768671504 us; error 56 us; skew 500 ppm
I20260809 22:49:28.671949 18645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:28 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315768, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:28.675129 18645 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:49:28.676792 18645 webserver.cc:460] Webserver started at http://127.24.136.62:37289/ using document root <none> and password file <none>
I20260809 22:49:28.677493 18645 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:28.677616 18645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:28.683207 18645 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:49:28.686478 18672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:28.688256 18645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:49:28.688455 18645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "dafdeacedc8846549a3e8eda31ef4808"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "873d0049ca377bff3b1ac20367c7a1e6"
server_key_iv: "e4c3a3267e491e26ebd9eb216aa0b41f"
server_key_version: "encryptionkey@0"
I20260809 22:49:28.689517 18645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:28.710096 18645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:28.714572 18645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:28.714851 18645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:28.733493 18645 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:41331
I20260809 22:49:28.733515 18771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:41331 every 8 connection(s)
I20260809 22:49:28.735105 18645 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:28.735670 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 18645
I20260809 22:49:28.736616 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:46611
--local_ip_for_outbound_sockets=127.24.136.1
--tserver_master_addrs=127.24.136.62:41331
--webserver_port=32913
--webserver_interface=127.24.136.1
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:28.742478 18772 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:49:28.746085 18772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Bootstrap starting.
I20260809 22:49:28.766989 18772 log.cc:824] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:28.783998 18772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:49:28.784538 18772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808: Bootstrap complete.
I20260809 22:49:28.793756 18772 raft_consensus.cc:348] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:28.794799 18772 raft_consensus.cc:729] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafdeacedc8846549a3e8eda31ef4808, State: Initialized, Role: FOLLOWER
I20260809 22:49:28.795408 18772 consensus_queue.cc:260] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:28.795670 18772 raft_consensus.cc:388] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:28.795783 18772 raft_consensus.cc:482] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:28.795969 18772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:49:28.799862 18772 raft_consensus.cc:504] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:28.800369 18772 leader_election.cc:302] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dafdeacedc8846549a3e8eda31ef4808; no voters:
I20260809 22:49:28.801095 18772 leader_election.cc:288] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 22:49:28.801131 18779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 22:49:28.802314 18779 raft_consensus.cc:686] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [term 3 LEADER]: Becoming Leader. State: Replica: dafdeacedc8846549a3e8eda31ef4808, State: Running, Role: LEADER
I20260809 22:49:28.802934 18779 consensus_queue.cc:237] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } }
I20260809 22:49:28.803264 18772 sys_catalog.cc:564] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:28.805648 18780 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafdeacedc8846549a3e8eda31ef4808. Latest consensus state: current_term: 3 leader_uuid: "dafdeacedc8846549a3e8eda31ef4808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } } }
I20260809 22:49:28.805683 18782 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "dafdeacedc8846549a3e8eda31ef4808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafdeacedc8846549a3e8eda31ef4808" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 41331 } } }
I20260809 22:49:28.806246 18780 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:28.806252 18782 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafdeacedc8846549a3e8eda31ef4808 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:28.809937 18787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:28.819621 18787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:28.821401 18787 catalog_manager.cc:1259] Loaded cluster ID: ff7cab745afb417fae825d13d0ec2a38
I20260809 22:49:28.821589 18787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 22:49:28.824005 18787 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f76412719ad google::LogMessage::Fail() at ??:0
@ 0x7f764127488a google::LogMessage::SendToLog() at ??:0
@ 0x7f76412714de google::LogMessage::Flush() at ??:0
@ 0x7f7641272a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7656ca4512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f7656ca1976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7656d29610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f7656d29439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7643ebd345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7643ec091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f7643ec0799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7643e9beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f76471756db start_thread at ??:0
@ 0x7f764010b61f clone at ??:0
W20260809 22:49:29.145136 18777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:29.145444 18777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:29.145581 18777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:29.153359 18777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:29.153506 18777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:29.153661 18777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:49:29.164386 18777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:46611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=32913
--tserver_master_addrs=127.24.136.62:41331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:29.165776 18777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:29.167239 18777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:29.181413 18843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:29.181458 18848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:29.181491 18846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:29.182159 18777 server_base.cc:1034] running on GCE node
I20260809 22:49:29.182873 18777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:29.183969 18777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:29.185149 18777 hybrid_clock.cc:648] HybridClock initialized: now 1786315769185084 us; error 53 us; skew 500 ppm
I20260809 22:49:29.185490 18777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:29 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315769, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:29.189033 18777 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:49:29.190685 18777 webserver.cc:460] Webserver started at http://127.24.136.1:32913/ using document root <none> and password file <none>
I20260809 22:49:29.191424 18777 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:29.191591 18777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:29.197575 18777 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:49:29.200793 18861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:29.202515 18777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:49:29.202740 18777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "05b73c39e1614c0d923a5cf51a02481c"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-pww0"
server_key: "5073dffc37cc1d3928d040a5ee98ff19"
server_key_iv: "3d28410d8cadc66eff1fd9077f9f72c4"
server_key_version: "encryptionkey@0"
I20260809 22:49:29.203900 18777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:29.235702 18777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:29.240476 18777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:29.240778 18777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:29.241904 18777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:29.243445 18777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:29.243599 18777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:49:29.243717 18777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:29.243798 18777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:29.277467 18777 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:46611
I20260809 22:49:29.277539 19047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:46611 every 8 connection(s)
I20260809 22:49:29.279008 18777 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:49:29.284329 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.282156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:35931)
0809 22:49:29.282944 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:29.283187 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":316,"threads_started":1}
W20260809 22:49:29.285081 19048 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:41331 (0 consecutive failures): Network error: Failed to ping master at 127.24.136.62:41331: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
I20260809 22:49:29.286020 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.285677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:46443)
0809 22:49:29.285832 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:29.285909 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 22:49:29.286926 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.286599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:58081)
0809 22:49:29.286720 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:29.286815 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 22:49:29.287211 19048 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:49:29.287695 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 18777
I20260809 22:49:29.288335 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:33379
--local_ip_for_outbound_sockets=127.24.136.2
--tserver_master_addrs=127.24.136.62:41331
--webserver_port=36501
--webserver_interface=127.24.136.2
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:29.640337 19052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:29.640655 19052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:29.640781 19052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:29.648218 19052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:29.648355 19052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:29.648514 19052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:49:29.658550 19052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:33379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=36501
--tserver_master_addrs=127.24.136.62:41331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:29.659979 19052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:29.661332 19052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:29.674516 19063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:29.674749 19061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:29.674458 19070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:29.675521 19052 server_base.cc:1034] running on GCE node
I20260809 22:49:29.676149 19052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:29.677146 19052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:29.678371 19052 hybrid_clock.cc:648] HybridClock initialized: now 1786315769678324 us; error 36 us; skew 500 ppm
I20260809 22:49:29.678714 19052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:29 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315769, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:29.681815 19052 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:49:29.683286 19052 webserver.cc:460] Webserver started at http://127.24.136.2:36501/ using document root <none> and password file <none>
I20260809 22:49:29.684002 19052 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:29.684121 19052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:29.689741 19052 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:49:29.692705 19082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:29.694097 19052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:49:29.694253 19052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "6275fa4eddb4407ab5b70fe4d795846b"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "7d455d2e5d77225e7d36c71f79fa8036"
server_key_iv: "573feddb4e5721b0a5f666d0112a8572"
server_key_version: "encryptionkey@0"
I20260809 22:49:29.695242 19052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:29.724766 19052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:29.729252 19052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:29.729497 19052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:29.730541 19052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:29.732167 19052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:29.732276 19052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:29.732363 19052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:29.732419 19052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:29.764613 19052 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:33379
I20260809 22:49:29.764720 19277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:33379 every 8 connection(s)
I20260809 22:49:29.766109 19052 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:49:29.768879 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 19052
I20260809 22:49:29.769539 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:32899
--local_ip_for_outbound_sockets=127.24.136.3
--tserver_master_addrs=127.24.136.62:41331
--webserver_port=39989
--webserver_interface=127.24.136.3
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:29.771375 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.769141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:35495)
0809 22:49:29.769985 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:29.770309 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":321,"threads_started":1}
W20260809 22:49:29.772194 19278 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:41331 (0 consecutive failures): Network error: Failed to ping master at 127.24.136.62:41331: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
I20260809 22:49:29.773131 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.772769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:44531)
0809 22:49:29.772934 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:29.773010 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 22:49:29.773985 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.773657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:48481)
0809 22:49:29.773799 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:29.773877 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:49:29.774212 19278 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:49:30.120822 19283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:30.121153 19283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:30.121250 19283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:30.128818 19283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:30.128938 19283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:30.129063 19283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:49:30.139441 19283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:32899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=39989
--tserver_master_addrs=127.24.136.62:41331
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:30.140853 19283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:30.142184 19283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:30.156011 19292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:30.156119 19293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:30.156533 19295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:30.156814 19283 server_base.cc:1034] running on GCE node
I20260809 22:49:30.157521 19283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:30.158562 19283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:30.159866 19283 hybrid_clock.cc:648] HybridClock initialized: now 1786315770159802 us; error 47 us; skew 500 ppm
I20260809 22:49:30.160246 19283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:30 dist-test-slave-pww0 krb5kdc[16807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315770, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:30.167882 19283 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:49:30.169569 19283 webserver.cc:460] Webserver started at http://127.24.136.3:39989/ using document root <none> and password file <none>
I20260809 22:49:30.170341 19283 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:30.170473 19283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:30.176456 19283 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:49:30.179683 19302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:30.181299 19283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:49:30.181509 19283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "17e4de2999c646c69f96a9a24b6ebf4a"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-pww0"
server_key: "90308d53c8857277821c76ec04383189"
server_key_iv: "796abb7af2ea80539698b2472c278d66"
server_key_version: "encryptionkey@0"
I20260809 22:49:30.182567 19283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:30.196936 19283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:30.201929 19283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:30.202229 19283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:30.203390 19283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:30.205210 19283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:30.205324 19283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:30.205439 19283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:30.205509 19283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:30.237725 19283 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:32899
I20260809 22:49:30.237778 19517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:32899 every 8 connection(s)
I20260809 22:49:30.239300 19283 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:49:30.240640 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 19283
I20260809 22:49:30.244743 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:30.242751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:52649)
0809 22:49:30.243587 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:30.243845 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":300,"threads_started":1}
W20260809 22:49:30.245477 19518 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:41331 (0 consecutive failures): Network error: Failed to ping master at 127.24.136.62:41331: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
I20260809 22:49:30.246471 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:30.246103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:60077)
0809 22:49:30.246262 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:30.246342 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 22:49:30.247411 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:30.247067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:42363)
0809 22:49:30.247198 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:30.247285 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:49:30.247691 19518 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:49:30.288775 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:30.287860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:38531)
0809 22:49:30.288414 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:30.288512 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":291,"threads_started":1}
I20260809 22:49:30.775949 19526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:30.774896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:53371)
0809 22:49:30.775517 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:30.775673 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":283,"threads_started":1}
I20260809 22:49:31.249432 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:31.248424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:37345)
0809 22:49:31.249038 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:31.249174 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":264,"threads_started":1}
I20260809 22:49:31.290650 19529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:31.289806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:47297)
0809 22:49:31.290371 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:31.290469 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":278,"threads_started":1}
I20260809 22:49:31.777714 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:31.776932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:53223)
0809 22:49:31.777434 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:31.777526 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":245,"threads_started":1}
I20260809 22:49:32.251402 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:32.250518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:41069)
0809 22:49:32.251091 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:32.251204 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":291,"threads_started":1}
I20260809 22:49:32.292577 19532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:32.291740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:37171)
0809 22:49:32.292270 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:32.292367 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":273,"threads_started":1}
I20260809 22:49:32.779505 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:32.778707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:53207)
0809 22:49:32.779187 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:32.779284 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":233,"threads_started":1}
I20260809 22:49:33.253472 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:33.252620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:58743)
0809 22:49:33.253169 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:33.253270 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":280,"threads_started":1}
I20260809 22:49:33.294562 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:33.293794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:54925)
0809 22:49:33.294285 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:33.294378 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":248,"threads_started":1}
I20260809 22:49:33.781188 19537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:33.780433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:49421)
0809 22:49:33.780929 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:33.781014 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":256,"threads_started":1}
I20260809 22:49:34.255450 19538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:34.254573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:47677)
0809 22:49:34.255110 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:34.255215 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":271,"threads_started":1}
I20260809 22:49:34.296507 19539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:34.295709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:35577)
0809 22:49:34.296223 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:34.296320 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":265,"threads_started":1}
I20260809 22:49:34.782944 19541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:34.782136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:60021)
0809 22:49:34.782660 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:34.782763 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":257,"threads_started":1}
I20260809 22:49:35.257418 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:35.256564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:48529)
0809 22:49:35.257124 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:35.257235 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":300,"threads_started":1}
I20260809 22:49:35.298448 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:35.297574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:55259)
0809 22:49:35.298157 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:35.298272 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":272,"threads_started":1}
I20260809 22:49:35.784848 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:35.784010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:43563)
0809 22:49:35.784557 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:35.784667 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":276,"threads_started":1}
I20260809 22:49:36.259307 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:36.258453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:41535)
0809 22:49:36.259019 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:36.259128 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":317,"threads_started":1}
I20260809 22:49:36.300730 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:36.299667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:41253)
0809 22:49:36.300290 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:36.300405 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":268,"threads_started":1}
I20260809 22:49:36.786893 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:36.785888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:46837)
0809 22:49:36.786555 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:36.786668 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":266,"threads_started":1}
I20260809 22:49:37.261433 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:37.260501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:53201)
0809 22:49:37.261088 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:37.261195 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":260,"threads_started":1}
I20260809 22:49:37.302624 19554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:37.301814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:53955)
0809 22:49:37.302351 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:37.302449 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":266,"threads_started":1}
I20260809 22:49:37.788720 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:37.787916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:43951)
0809 22:49:37.788448 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:37.788545 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":262,"threads_started":1}
I20260809 22:49:38.263274 19557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:38.262454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:35835)
0809 22:49:38.262987 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:38.263081 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":291,"threads_started":1}
I20260809 22:49:38.304699 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:38.303765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:48317)
0809 22:49:38.304333 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:38.304496 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":265,"threads_started":1}
I20260809 22:49:38.790482 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:38.789672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:52287)
0809 22:49:38.790180 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:38.790294 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":250,"threads_started":1}
I20260809 22:49:39.265061 19562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:39.264222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:34633)
0809 22:49:39.264784 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:39.264879 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":268,"threads_started":1}
I20260809 22:49:39.306594 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:39.305850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:52961)
0809 22:49:39.306332 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:39.306421 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":259,"threads_started":1}
I20260809 22:49:39.792419 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:39.791595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:44747)
0809 22:49:39.792118 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:39.792218 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":271,"threads_started":1}
I20260809 22:49:40.266978 19568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:40.266152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:34703)
0809 22:49:40.266700 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:40.266804 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":289,"threads_started":1}
I20260809 22:49:40.308521 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:40.307716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:51399)
0809 22:49:40.308213 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:40.308326 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":234,"threads_started":1}
I20260809 22:49:40.794173 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:40.793420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:36169)
0809 22:49:40.793879 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:40.793984 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":240,"threads_started":1}
I20260809 22:49:41.269068 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:41.268112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:50245)
0809 22:49:41.268706 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:41.268844 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":260,"threads_started":1}
I20260809 22:49:41.310317 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:41.309507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:40485)
0809 22:49:41.310014 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:41.310113 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":249,"threads_started":1}
I20260809 22:49:41.796079 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:41.795232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:42835)
0809 22:49:41.795809 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:41.795910 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":318,"threads_started":1}
I20260809 22:49:42.270920 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:42.270107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:35805)
0809 22:49:42.270635 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:42.270734 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":274,"threads_started":1}
I20260809 22:49:42.312346 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:42.311450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:41653)
0809 22:49:42.312031 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:42.312144 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":269,"threads_started":1}
I20260809 22:49:42.798017 19582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:42.797145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:47073)
0809 22:49:42.797714 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:42.797822 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":269,"threads_started":1}
I20260809 22:49:43.272764 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:43.271891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:39877)
0809 22:49:43.272432 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:43.272542 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":252,"threads_started":1}
I20260809 22:49:43.314203 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:43.313389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:43025)
0809 22:49:43.313901 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:43.313994 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":250,"threads_started":1}
I20260809 22:49:43.799929 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:43.799022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:53403)
0809 22:49:43.799604 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:43.799712 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":284,"threads_started":1}
I20260809 22:49:44.274507 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:44.273689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:38429)
0809 22:49:44.274209 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:44.274306 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":243,"threads_started":1}
I20260809 22:49:44.316380 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:44.315580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.1:43239)
0809 22:49:44.316068 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:44.316167 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":231,"threads_started":1}
I20260809 22:49:44.801929 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:44.801114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.2:42689)
0809 22:49:44.801625 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:44.801745 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":244,"threads_started":1}
I20260809 22:49:45.248360 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 18777
I20260809 22:49:45.274609 19599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:45.249705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43136 (local address 127.24.136.1:46611)
0809 22:49:45.250318 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:45.250340 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:45.250395 (+ 55us) server_negotiation.cc:407] Connection header received
0809 22:49:45.252081 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:45.252093 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:45.252547 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:45.252916 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:45.254541 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:45.256626 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:45.258448 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:45.259691 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:45.259920 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:45.266289 (+ 6369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:45.266304 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:45.266330 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:45.266399 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:45.270545 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:45.271497 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:45.271509 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:45.271520 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:45.271791 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:45.272181 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:45.272186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:45.272190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:45.272986 (+ 796us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:45.273333 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:45.273665 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 22:49:45.274181 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":594,"threads_started":1}
I20260809 22:49:45.276497 19600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:45.275648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:41331 (local address 127.24.136.3:59451)
0809 22:49:45.276131 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:45.276259 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:41331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":246,"threads_started":1}
I20260809 22:49:45.276643 19040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:45.369107 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 18777
I20260809 22:49:45.393749 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 19052
I20260809 22:49:45.412715 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:45.394705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.24.136.2:33379)
0809 22:49:45.395273 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:45.395289 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:45.395330 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:49:45.396972 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:45.396983 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:45.397450 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:45.397799 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:45.399417 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:45.400765 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:45.402071 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:45.402797 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:45.402985 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:45.405888 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:45.405897 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:45.405916 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:45.405965 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:45.409067 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:45.409853 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:45.409861 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:45.409869 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:45.410062 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:45.410496 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:45.410500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:45.410502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:45.411080 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:45.411396 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:45.411958 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 22:49:45.412418 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":248,"threads_started":1}
I20260809 22:49:45.414369 19271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:45.498988 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 19052
I20260809 22:49:45.529474 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 19283
I20260809 22:49:45.549252 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:45.530757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.24.136.3:32899)
0809 22:49:45.531378 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:45.531395 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:45.531440 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:49:45.533230 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:45.533240 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:45.533705 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:45.534018 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:45.535609 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:45.536958 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:45.538381 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:45.539147 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:45.539339 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:45.542295 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:45.542304 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:45.542324 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:45.542377 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:45.545448 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:45.546276 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:45.546284 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:45.546292 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:45.546491 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:45.546932 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:45.546937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:45.546939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:45.547629 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:45.547948 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:45.548424 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 22:49:45.548920 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":283,"threads_started":1}
I20260809 22:49:45.550851 19511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:45.635842 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 19283
2026-08-09T22:49:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25570 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:45 dist-test-slave-pww0 krb5kdc[19613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:45 dist-test-slave-pww0 krb5kdc[19613](info): set up 2 sockets
Aug 09 22:49:45 dist-test-slave-pww0 krb5kdc[19613](info): commencing operation
krb5kdc: starting...
W20260809 22:49:46.905158 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:49:47 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:49:47.211961 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38989
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:47.575443 19635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:47.575815 19635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:47.575960 19635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:47.583575 19635 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 22:49:47.583690 19635 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 22:49:47.583738 19635 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 22:49:47.583820 19635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:47.583896 19635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:47.583992 19635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:47.584057 19635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:47.584133 19635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:47.594569 19635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:47.595999 19635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:47.597399 19635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:47.610110 19641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:47.610050 19642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:47.610080 19646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:47.610929 19635 server_base.cc:1034] running on GCE node
I20260809 22:49:47.611833 19635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:47.613350 19635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:47.614563 19635 hybrid_clock.cc:648] HybridClock initialized: now 1786315787614484 us; error 55 us; skew 500 ppm
I20260809 22:49:47.614905 19635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:47 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315787, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:47.618288 19635 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:49:47.620000 19635 webserver.cc:460] Webserver started at http://127.24.136.62:40915/ using document root <none> and password file <none>
I20260809 22:49:47.620726 19635 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:47.620844 19635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:47.621174 19635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:47.624403 19635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "14cf8de914614d39903c45fff959399a"
format_stamp: "Formatted at 2026-08-09 22:49:47 on dist-test-slave-pww0"
server_key: "b4aa4289808e2d584bd42f3d455b06b4"
server_key_iv: "f6c24ebef10d3f2fdb0ddca8f32feb31"
server_key_version: "encryptionkey@0"
I20260809 22:49:47.625219 19635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "14cf8de914614d39903c45fff959399a"
format_stamp: "Formatted at 2026-08-09 22:49:47 on dist-test-slave-pww0"
server_key: "b4aa4289808e2d584bd42f3d455b06b4"
server_key_iv: "f6c24ebef10d3f2fdb0ddca8f32feb31"
server_key_version: "encryptionkey@0"
I20260809 22:49:47.630625 19635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:49:47.634426 19656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:47.635890 19635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:49:47.636073 19635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "14cf8de914614d39903c45fff959399a"
format_stamp: "Formatted at 2026-08-09 22:49:47 on dist-test-slave-pww0"
server_key: "b4aa4289808e2d584bd42f3d455b06b4"
server_key_iv: "f6c24ebef10d3f2fdb0ddca8f32feb31"
server_key_version: "encryptionkey@0"
I20260809 22:49:47.636332 19635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:47.667876 19635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:47.672465 19635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:47.672772 19635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:47.691788 19635 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38989
I20260809 22:49:47.691809 19747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38989 every 8 connection(s)
I20260809 22:49:47.693316 19635 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:47.697559 19748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:47.703585 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 19635
I20260809 22:49:47.703814 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:49:47.704468 25120 external_mini_cluster.cc:1428] Setting key 9e8068a3aaa4077261fe05176f712c9e
I20260809 22:49:47.709334 19748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Bootstrap starting.
I20260809 22:49:47.713279 19748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:47.714610 19748 log.cc:824] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:47.717928 19748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: No bootstrap required, opened a new log
Aug 09 22:49:47 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:47.726562 19748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:49:47.726994 19748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:47.727106 19748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14cf8de914614d39903c45fff959399a, State: Initialized, Role: FOLLOWER
I20260809 22:49:47.727782 19748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:49:47.728041 19748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:47.728180 19748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:47.728358 19748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:47.729600 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:47.706702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46508 (local address 127.24.136.62:38989)
0809 22:49:47.707591 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:47.707604 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:47.707650 (+ 46us) server_negotiation.cc:407] Connection header received
0809 22:49:47.709411 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:47.709437 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:47.709894 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:47.710238 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:47.712240 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:47.713681 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:47.715540 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:47.716280 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:47.716463 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:47.721184 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:47.721196 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:47.721215 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:47.721277 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:47.724971 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:47.725859 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:47.725867 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:47.725875 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:47.726074 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:47.726501 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:47.726505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:47.726507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:47.727191 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:47.727555 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:47.728022 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 22:49:47.728528 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":381,"threads_started":1}
I20260809 22:49:47.730939 19748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:49:47.731580 19748 leader_election.cc:302] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14cf8de914614d39903c45fff959399a; no voters:
I20260809 22:49:47.732298 19748 leader_election.cc:288] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:47.732401 19756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:47.733528 19756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 1 LEADER]: Becoming Leader. State: Replica: 14cf8de914614d39903c45fff959399a, State: Running, Role: LEADER
I20260809 22:49:47.734047 19756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:49:47.734517 19748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:47.737138 19758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14cf8de914614d39903c45fff959399a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } } }
I20260809 22:49:47.737411 19759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14cf8de914614d39903c45fff959399a. Latest consensus state: current_term: 1 leader_uuid: "14cf8de914614d39903c45fff959399a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } } }
I20260809 22:49:47.737617 19758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:47.737697 19759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:47.740371 19771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:47.745342 19771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:47.754669 19771 catalog_manager.cc:1347] Generated new cluster ID: 87711c6231e64362b3a2b5211732697c
I20260809 22:49:47.754818 19771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:47.766709 19771 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:47.768782 19771 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:47.775853 19771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Generated new TSK 0
I20260809 22:49:47.776765 19771 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:49:47.858618 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38989
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:48.243716 19788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:48.244029 19788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:48.244169 19788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:48.251765 19788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:48.251911 19788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:48.252063 19788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:49:48.262264 19788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:48.263756 19788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:48.265125 19788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:48.278468 19795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:48.278798 19794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:48.278569 19797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:48.279547 19788 server_base.cc:1034] running on GCE node
I20260809 22:49:48.280229 19788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:48.281280 19788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:48.282491 19788 hybrid_clock.cc:648] HybridClock initialized: now 1786315788282440 us; error 49 us; skew 500 ppm
I20260809 22:49:48.282823 19788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:48 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315788, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:48.286113 19788 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:49:48.287729 19788 webserver.cc:460] Webserver started at http://127.24.136.1:34819/ using document root <none> and password file <none>
I20260809 22:49:48.288460 19788 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:48.288586 19788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:48.288911 19788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:48.292088 19788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "13d1aa2ed32e4f07ba96dbd8113b8952"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "523e2fce95f9803eae39afd616b9aaae"
server_key_iv: "15dfd8dea66b648a6477c63278dd4902"
server_key_version: "encryptionkey@0"
I20260809 22:49:48.292891 19788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "13d1aa2ed32e4f07ba96dbd8113b8952"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "523e2fce95f9803eae39afd616b9aaae"
server_key_iv: "15dfd8dea66b648a6477c63278dd4902"
server_key_version: "encryptionkey@0"
I20260809 22:49:48.298382 19788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:49:48.302091 19809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:48.303571 19788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:49:48.303752 19788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "13d1aa2ed32e4f07ba96dbd8113b8952"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "523e2fce95f9803eae39afd616b9aaae"
server_key_iv: "15dfd8dea66b648a6477c63278dd4902"
server_key_version: "encryptionkey@0"
I20260809 22:49:48.303958 19788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:48.333392 19788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:48.338207 19788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:48.338527 19788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:48.339701 19788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:48.341398 19788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:48.341490 19788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:48.341637 19788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:48.341693 19788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:48.375087 19788 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:44493
I20260809 22:49:48.375133 20001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:44493 every 8 connection(s)
I20260809 22:49:48.376598 19788 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:49:48.386065 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 19788
I20260809 22:49:48.386288 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:49:48.386976 25120 external_mini_cluster.cc:1428] Setting key 781405e4bfd3aa14841385fc3c938084
Aug 09 22:49:48 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315788, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:48.398440 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:48.379385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:36527 (local address 127.24.136.62:38989)
0809 22:49:48.379897 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:48.379902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:48.380928 (+ 1026us) server_negotiation.cc:407] Connection header received
0809 22:49:48.382891 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:48.382897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:48.382983 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:48.383086 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:48.384829 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:48.385587 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:48.387547 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:48.388101 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:48.388252 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:48.392271 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:48.392277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:48.392280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:48.392319 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:48.394920 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:48.395974 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:48.395979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:48.395982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:48.396123 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:48.396628 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:48.396631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:48.396633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:48.396924 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:48.397041 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:48.397971 (+ 930us) server_negotiation.cc:299] Negotiation successful
0809 22:49:48.398167 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":250,"threads_started":1}
I20260809 22:49:48.399830 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:48.380031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:36527)
0809 22:49:48.380748 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:48.380801 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:48.382552 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:48.383370 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:48.383381 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:48.383985 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:48.384618 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:48.384637 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:48.385798 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:48.385803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:48.387333 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:48.387350 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:48.388385 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:48.388394 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:48.388648 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:48.389350 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:48.389377 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:48.391980 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:48.395092 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:48.395102 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:48.395123 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:48.395820 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:48.396234 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:48.396238 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:48.396241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:48.396527 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:48.397048 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:48.397056 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:48.397528 (+ 472us) client_negotiation.cc:769] Sending connection context
0809 22:49:48.397847 (+ 319us) client_negotiation.cc:240] Negotiation successful
0809 22:49:48.398408 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":269,"threads_started":1}
I20260809 22:49:48.401522 20002 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38989
I20260809 22:49:48.401854 20002 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:48.402571 20002 heartbeater.cc:511] Master 127.24.136.62:38989 requested a full tablet report, sending...
I20260809 22:49:48.405177 19685 ts_manager.cc:194] Registered new tserver with Master: 13d1aa2ed32e4f07ba96dbd8113b8952 (127.24.136.1:44493)
I20260809 22:49:48.407234 19685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:36527
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:49:48.455912 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38989
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:48.823326 20014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:48.823724 20014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:48.823843 20014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:48.831321 20014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:48.831487 20014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:48.831637 20014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:49:48.842315 20014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:48.843700 20014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:48.845152 20014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:48.859412 20020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:48.859364 20019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:48.859483 20022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:48.860234 20014 server_base.cc:1034] running on GCE node
I20260809 22:49:48.860924 20014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:48.861984 20014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:48.863224 20014 hybrid_clock.cc:648] HybridClock initialized: now 1786315788863159 us; error 56 us; skew 500 ppm
I20260809 22:49:48.863648 20014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:48 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315788, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:48.867111 20014 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:49:48.868753 20014 webserver.cc:460] Webserver started at http://127.24.136.2:34511/ using document root <none> and password file <none>
I20260809 22:49:48.869495 20014 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:48.869625 20014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:48.869946 20014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:48.873112 20014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "3a5ee0af3c8d4da984658c7dc26a657c"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "bda3acbe6e6ada2c9cefa5863d989436"
server_key_iv: "7bb0e993b1b7899d8876b4484ed39e60"
server_key_version: "encryptionkey@0"
I20260809 22:49:48.874010 20014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "3a5ee0af3c8d4da984658c7dc26a657c"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "bda3acbe6e6ada2c9cefa5863d989436"
server_key_iv: "7bb0e993b1b7899d8876b4484ed39e60"
server_key_version: "encryptionkey@0"
I20260809 22:49:48.879834 20014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260809 22:49:48.883535 20043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:48.885010 20014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:49:48.885188 20014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "3a5ee0af3c8d4da984658c7dc26a657c"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "bda3acbe6e6ada2c9cefa5863d989436"
server_key_iv: "7bb0e993b1b7899d8876b4484ed39e60"
server_key_version: "encryptionkey@0"
I20260809 22:49:48.885416 20014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:48.908264 20014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:48.912776 20014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:48.913058 20014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:48.914114 20014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:48.915589 20014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:48.915666 20014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:48.915782 20014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:48.915831 20014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:48.948246 20014 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:46755
I20260809 22:49:48.948302 20232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:46755 every 8 connection(s)
I20260809 22:49:48.949611 20014 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:49:48.957482 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 20014
I20260809 22:49:48.957679 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:49:48.958134 25120 external_mini_cluster.cc:1428] Setting key 978986944440f006b6c58fac17b2be1c
Aug 09 22:49:48 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315788, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:48.972965 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:48.952554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:39663 (local address 127.24.136.62:38989)
0809 22:49:48.953082 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:48.953087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:48.954211 (+ 1124us) server_negotiation.cc:407] Connection header received
0809 22:49:48.956313 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:48.956319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:48.956415 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:48.956530 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:48.958387 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:48.959359 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:48.961564 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:48.962195 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:48.962335 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:48.966266 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:48.966273 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:48.966278 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:48.966322 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:48.969536 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:48.970420 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:48.970426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:48.970429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:48.970603 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:48.971048 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:48.971052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:48.971055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:48.971425 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:48.971608 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:48.972360 (+ 752us) server_negotiation.cc:299] Negotiation successful
0809 22:49:48.972593 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":259,"threads_started":1}
I20260809 22:49:48.973919 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:48.953177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:39663)
0809 22:49:48.954010 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:48.954061 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:48.955972 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:48.956838 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:48.956847 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:48.957442 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:48.958149 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:48.958170 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:48.959589 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:48.959595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:48.961370 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:48.961388 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:48.962448 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:48.962453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:48.962694 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:48.963429 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:48.963515 (+ 86us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:48.965964 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:48.969726 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:48.969735 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:48.969751 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:48.970261 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:48.970721 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:48.970724 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:48.970726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:48.970936 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:48.971587 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:48.971594 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:48.971977 (+ 383us) client_negotiation.cc:769] Sending connection context
0809 22:49:48.972222 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 22:49:48.972687 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":315,"threads_started":1}
I20260809 22:49:48.975587 20233 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38989
I20260809 22:49:48.975919 20233 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:48.976703 20233 heartbeater.cc:511] Master 127.24.136.62:38989 requested a full tablet report, sending...
I20260809 22:49:48.978451 19685 ts_manager.cc:194] Registered new tserver with Master: 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
I20260809 22:49:48.979565 19685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:39663
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:49:49.041241 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38989
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:49.400539 20240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:49.400869 20240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:49.401010 20240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:49.408731 20240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:49.408872 20240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:49.409025 20240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:49:49.410735 20002 heartbeater.cc:503] Master 127.24.136.62:38989 was elected leader, sending a full tablet report...
I20260809 22:49:49.419886 20240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:49:49.421315 20240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:49.422756 20240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:49.436765 20248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:49.436795 20245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:49.436834 20246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:49.437587 20240 server_base.cc:1034] running on GCE node
I20260809 22:49:49.438211 20240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:49.439272 20240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:49.440477 20240 hybrid_clock.cc:648] HybridClock initialized: now 1786315789440406 us; error 57 us; skew 500 ppm
I20260809 22:49:49.440886 20240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:49 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315789, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:49.444339 20240 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:49:49.446033 20240 webserver.cc:460] Webserver started at http://127.24.136.3:40319/ using document root <none> and password file <none>
I20260809 22:49:49.446796 20240 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:49.446910 20240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:49.447239 20240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:49.450484 20240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "78eb401e40f34f7eb81022428aef56d4"
format_stamp: "Formatted at 2026-08-09 22:49:49 on dist-test-slave-pww0"
server_key: "d199bb72824a7ab30b6b6e952d2df832"
server_key_iv: "937679f8720373dbd2302238fd6bb6d5"
server_key_version: "encryptionkey@0"
I20260809 22:49:49.451367 20240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "78eb401e40f34f7eb81022428aef56d4"
format_stamp: "Formatted at 2026-08-09 22:49:49 on dist-test-slave-pww0"
server_key: "d199bb72824a7ab30b6b6e952d2df832"
server_key_iv: "937679f8720373dbd2302238fd6bb6d5"
server_key_version: "encryptionkey@0"
I20260809 22:49:49.457412 20240 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 22:49:49.461602 20261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:49.463259 20240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:49:49.463485 20240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "78eb401e40f34f7eb81022428aef56d4"
format_stamp: "Formatted at 2026-08-09 22:49:49 on dist-test-slave-pww0"
server_key: "d199bb72824a7ab30b6b6e952d2df832"
server_key_iv: "937679f8720373dbd2302238fd6bb6d5"
server_key_version: "encryptionkey@0"
I20260809 22:49:49.463708 20240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:49.481673 20240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:49.486297 20240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:49.486603 20240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:49.487735 20240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:49:49.489322 20240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:49:49.489432 20240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:49.489569 20240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:49:49.489630 20240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:49.526314 20240 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:43681
I20260809 22:49:49.526352 20457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:43681 every 8 connection(s)
I20260809 22:49:49.527737 20240 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:49:49.533607 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 20240
I20260809 22:49:49.533826 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:49:49.534420 25120 external_mini_cluster.cc:1428] Setting key fbb39158a8605099214144bf0707d218
Aug 09 22:49:49 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315789, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:49:49.550902 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.530404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:58739 (local address 127.24.136.62:38989)
0809 22:49:49.530841 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:49.530846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:49.531902 (+ 1056us) server_negotiation.cc:407] Connection header received
0809 22:49:49.534141 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:49.534146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:49.534223 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:49.534392 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:49.536600 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.537456 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.539235 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.539707 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.539829 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.543718 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:49.543724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:49.543744 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:49.543792 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:49.546770 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:49.547786 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:49.547791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:49.547793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:49.547948 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:49.548617 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:49.548624 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:49.548627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:49.549039 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:49.549229 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:49.550219 (+ 990us) server_negotiation.cc:299] Negotiation successful
0809 22:49:49.550510 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":216,"threads_started":1}
I20260809 22:49:49.551862 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.530899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:58739)
0809 22:49:49.531727 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:49.531769 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:49.533794 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:49.534737 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:49.534749 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:49.535569 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:49.536400 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.536432 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.537625 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.537636 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.539078 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.539093 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.540089 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.540094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.540325 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.541045 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:49.541070 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:49.543449 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:49.546982 (+ 3533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:49.546991 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:49.547009 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:49.547611 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:49.548067 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:49.548071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:49.548074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:49.548401 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:49.549206 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:49.549215 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:49.549729 (+ 514us) client_negotiation.cc:769] Sending connection context
0809 22:49:49.550038 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 22:49:49.550585 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":304,"threads_started":1}
I20260809 22:49:49.553293 20461 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38989
I20260809 22:49:49.553586 20461 heartbeater.cc:464] Registering TS with master...
I20260809 22:49:49.554332 20461 heartbeater.cc:511] Master 127.24.136.62:38989 requested a full tablet report, sending...
I20260809 22:49:49.555866 19685 ts_manager.cc:194] Registered new tserver with Master: 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681)
I20260809 22:49:49.556692 19685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:58739
I20260809 22:49:49.563656 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:49:49.582971 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.568506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46516 (local address 127.24.136.62:38989)
0809 22:49:49.568724 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:49.568729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:49.569255 (+ 526us) server_negotiation.cc:407] Connection header received
0809 22:49:49.569429 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:49.569433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:49.569506 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:49.569598 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:49.571067 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.571920 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.573779 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.574227 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.574352 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.577015 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:49.577021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:49.577024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:49.577069 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:49.579830 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:49.580740 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:49.580745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:49.580747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:49.580925 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:49.581361 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:49.581365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:49.581367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:49.581746 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:49.582015 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:49.582368 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 22:49:49.582630 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:49:49.587579 19685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:49:49.595033 19685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:49:49.630224 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.609657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40928 (local address 127.24.136.2:46755)
0809 22:49:49.621563 (+ 11906us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:49.621574 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:49.621608 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:49:49.621730 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:49.621739 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:49.622047 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:49.622337 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:49.623136 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.624687 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.627530 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.629025 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.629176 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.629376 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:49.629748 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 22:49:49.629919 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11683,"thread_start_us":11784,"threads_started":1}
I20260809 22:49:49.630223 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.609020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:38884)
0809 22:49:49.620802 (+ 11782us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:49.620825 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:49.620982 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:49.622697 (+ 1715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:49.622703 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:49.622737 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:49.623188 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.623200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.624987 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.624993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.626851 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.626869 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.629068 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.629074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.629319 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.629391 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 22:49:49.629572 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 22:49:49.629796 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11603,"mutex_wait_us":28,"thread_start_us":10184,"threads_started":1}
I20260809 22:49:49.630825 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.609395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:46755 (local address 127.0.0.1:40928)
0809 22:49:49.621145 (+ 11750us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:49.621165 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:49.621298 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:49.622350 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:49.622359 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:49.622393 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:49.622871 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.622888 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.624845 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.624851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.627297 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.627313 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.629256 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.629263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.629452 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.629478 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:49:49.629561 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 22:49:49.630553 (+ 992us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11614,"mutex_wait_us":823,"thread_start_us":11037,"threads_started":1}
I20260809 22:49:49.631528 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.609703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38884 (local address 127.24.136.1:44493)
0809 22:49:49.622076 (+ 12373us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:49.622082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:49.622107 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:49:49.622185 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:49.622191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:49.622431 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:49.622659 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:49.623382 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.624841 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.627034 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.628542 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.628698 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.628940 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:49.629654 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 22:49:49.629854 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12218,"thread_start_us":12060,"threads_started":1}
I20260809 22:49:49.632583 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.608767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:43681 (local address 127.0.0.1:39072)
0809 22:49:49.618616 (+ 9849us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:49.618695 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:49.618958 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:49.624515 (+ 5557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:49.624520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:49.624544 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:49.625251 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.625265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.627390 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.627395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.629567 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.629585 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.631848 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.631854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.632025 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.632059 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:49:49.632147 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:49:49.632273 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9457,"mutex_wait_us":46,"thread_start_us":12485,"threads_started":1}
I20260809 22:49:49.635210 19893 tablet_service.cc:1467] Processing CreateTablet for tablet 5ef7b15e417143d597bb160f00cc09d7 (DEFAULT_TABLE table=test-table [id=824a538612d24995a2e85910059e5c9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:49.635221 20124 tablet_service.cc:1467] Processing CreateTablet for tablet 5ef7b15e417143d597bb160f00cc09d7 (DEFAULT_TABLE table=test-table [id=824a538612d24995a2e85910059e5c9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:49.636315 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.609074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39072 (local address 127.24.136.3:43681)
0809 22:49:49.623714 (+ 14640us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:49.623722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:49.623753 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:49:49.623844 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:49.623851 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:49.624067 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:49.624262 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:49.625532 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.627206 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.629787 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.631263 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.631389 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.631641 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:49.635729 (+ 4088us) server_negotiation.cc:299] Negotiation successful
0809 22:49:49.635944 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14443,"thread_start_us":13950,"threads_started":1}
I20260809 22:49:49.636852 20124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ef7b15e417143d597bb160f00cc09d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:49.636874 19893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ef7b15e417143d597bb160f00cc09d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:49.640741 20339 tablet_service.cc:1467] Processing CreateTablet for tablet 5ef7b15e417143d597bb160f00cc09d7 (DEFAULT_TABLE table=test-table [id=824a538612d24995a2e85910059e5c9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:49:49.642469 20339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ef7b15e417143d597bb160f00cc09d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:49.650116 20493 tablet_bootstrap.cc:492] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: Bootstrap starting.
I20260809 22:49:49.652491 20494 tablet_bootstrap.cc:492] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: Bootstrap starting.
I20260809 22:49:49.655037 20493 tablet_bootstrap.cc:654] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:49.656747 20494 tablet_bootstrap.cc:654] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:49.656829 20493 log.cc:824] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:49.657466 20495 tablet_bootstrap.cc:492] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: Bootstrap starting.
I20260809 22:49:49.658131 20494 log.cc:824] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:49.661105 20495 tablet_bootstrap.cc:654] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:49.661283 20493 tablet_bootstrap.cc:492] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: No bootstrap required, opened a new log
I20260809 22:49:49.661556 20493 ts_tablet_manager.cc:1397] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260809 22:49:49.662142 20495 log.cc:824] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:49.665148 20494 tablet_bootstrap.cc:492] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: No bootstrap required, opened a new log
I20260809 22:49:49.665436 20495 tablet_bootstrap.cc:492] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: No bootstrap required, opened a new log
I20260809 22:49:49.665457 20494 ts_tablet_manager.cc:1397] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260809 22:49:49.665719 20495 ts_tablet_manager.cc:1397] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260809 22:49:49.672361 20493 raft_consensus.cc:348] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:49:49.672927 20493 raft_consensus.cc:374] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:49.673044 20493 raft_consensus.cc:729] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5ee0af3c8d4da984658c7dc26a657c, State: Initialized, Role: FOLLOWER
I20260809 22:49:49.673087 20494 raft_consensus.cc:348] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:49:49.673492 20494 raft_consensus.cc:374] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:49.673599 20494 raft_consensus.cc:729] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13d1aa2ed32e4f07ba96dbd8113b8952, State: Initialized, Role: FOLLOWER
I20260809 22:49:49.673763 20493 consensus_queue.cc:260] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:49:49.674165 20494 consensus_queue.cc:260] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:49:49.676532 20495 raft_consensus.cc:348] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:49:49.677030 20495 raft_consensus.cc:374] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:49.677134 20495 raft_consensus.cc:729] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78eb401e40f34f7eb81022428aef56d4, State: Initialized, Role: FOLLOWER
I20260809 22:49:49.677627 20233 heartbeater.cc:503] Master 127.24.136.62:38989 was elected leader, sending a full tablet report...
I20260809 22:49:49.677939 20493 ts_tablet_manager.cc:1428] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260809 22:49:49.677775 20495 consensus_queue.cc:260] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:49:49.678882 20461 heartbeater.cc:503] Master 127.24.136.62:38989 was elected leader, sending a full tablet report...
I20260809 22:49:49.680981 20494 ts_tablet_manager.cc:1428] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: Time spent starting tablet: real 0.015s user 0.005s sys 0.008s
I20260809 22:49:49.681425 20495 ts_tablet_manager.cc:1428] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
W20260809 22:49:49.701846 20234 tablet.cc:2369] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:49.779397 20507 raft_consensus.cc:482] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 22:49:49.779735 20462 tablet.cc:2369] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:49.779832 20507 raft_consensus.cc:504] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:49:49.781754 20507 leader_election.cc:288] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13d1aa2ed32e4f07ba96dbd8113b8952 (127.24.136.1:44493), 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
I20260809 22:49:49.788089 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.782222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:43671 (local address 127.24.136.1:44493)
0809 22:49:49.782459 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:49.782465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:49.782532 (+ 67us) server_negotiation.cc:407] Connection header received
0809 22:49:49.782805 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:49.782812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:49.782897 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:49.783006 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:49.783400 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.784406 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.786214 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.787264 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.787399 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.787485 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:49.787799 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 22:49:49.787894 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 22:49:49.788215 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.782046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.24.136.3:43671)
0809 22:49:49.782447 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:49.782467 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:49.782623 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:49.783026 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:49.783030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:49.783059 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:49.783246 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.783258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.784545 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.784551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.786039 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.786052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.787415 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.787422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.787640 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.787679 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:49:49.787781 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 22:49:49.787921 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":1}
I20260809 22:49:49.789170 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.782164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:46755 (local address 127.24.136.3:53363)
0809 22:49:49.783075 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:49.783092 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:49.783216 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:49.783571 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:49.783576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:49.783602 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:49:49.783991 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.784004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.785201 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.785205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.786796 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:49.786812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.788513 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.788521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:49.788689 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.788722 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:49:49.788829 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 22:49:49.788924 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":529,"threads_started":1}
I20260809 22:49:49.789355 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.782420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53363 (local address 127.24.136.2:46755)
0809 22:49:49.782609 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:49.782615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:49.783202 (+ 587us) server_negotiation.cc:407] Connection header received
0809 22:49:49.783347 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:49.783352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:49.783424 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:49.783536 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:49:49.784132 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.785036 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.786964 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.788015 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.788173 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.788246 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:49.788885 (+ 639us) server_negotiation.cc:299] Negotiation successful
0809 22:49:49.789022 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:49:49.789373 19922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ef7b15e417143d597bb160f00cc09d7" candidate_uuid: "78eb401e40f34f7eb81022428aef56d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" is_pre_election: true
I20260809 22:49:49.790043 19922 raft_consensus.cc:2454] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78eb401e40f34f7eb81022428aef56d4 in term 0.
I20260809 22:49:49.790319 20151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ef7b15e417143d597bb160f00cc09d7" candidate_uuid: "78eb401e40f34f7eb81022428aef56d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" is_pre_election: true
I20260809 22:49:49.790764 20271 leader_election.cc:302] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13d1aa2ed32e4f07ba96dbd8113b8952, 78eb401e40f34f7eb81022428aef56d4; no voters:
I20260809 22:49:49.790889 20151 raft_consensus.cc:2454] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78eb401e40f34f7eb81022428aef56d4 in term 0.
I20260809 22:49:49.791173 20507 raft_consensus.cc:2781] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:49:49.791332 20507 raft_consensus.cc:482] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:49:49.791446 20507 raft_consensus.cc:3037] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:49.793778 20507 raft_consensus.cc:504] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:49:49.794430 20507 leader_election.cc:288] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [CANDIDATE]: Term 1 election: Requested vote from peers 13d1aa2ed32e4f07ba96dbd8113b8952 (127.24.136.1:44493), 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
I20260809 22:49:49.795058 20151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ef7b15e417143d597bb160f00cc09d7" candidate_uuid: "78eb401e40f34f7eb81022428aef56d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5ee0af3c8d4da984658c7dc26a657c"
I20260809 22:49:49.795059 19922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ef7b15e417143d597bb160f00cc09d7" candidate_uuid: "78eb401e40f34f7eb81022428aef56d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952"
I20260809 22:49:49.795315 20151 raft_consensus.cc:3037] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:49.795303 19922 raft_consensus.cc:3037] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:49.798659 19922 raft_consensus.cc:2454] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78eb401e40f34f7eb81022428aef56d4 in term 1.
I20260809 22:49:49.798659 20151 raft_consensus.cc:2454] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78eb401e40f34f7eb81022428aef56d4 in term 1.
I20260809 22:49:49.799213 20271 leader_election.cc:302] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 13d1aa2ed32e4f07ba96dbd8113b8952, 3a5ee0af3c8d4da984658c7dc26a657c, 78eb401e40f34f7eb81022428aef56d4; no voters:
I20260809 22:49:49.799600 20507 raft_consensus.cc:2781] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:49.800058 20507 raft_consensus.cc:686] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 1 LEADER]: Becoming Leader. State: Replica: 78eb401e40f34f7eb81022428aef56d4, State: Running, Role: LEADER
I20260809 22:49:49.800590 20507 consensus_queue.cc:237] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:49:49.805384 19685 catalog_manager.cc:5495] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "78eb401e40f34f7eb81022428aef56d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } health_report { overall_health: UNKNOWN } } }
I20260809 22:49:49.831255 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.824864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.24.136.3:43681)
0809 22:49:49.825117 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:49.825123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:49.825178 (+ 55us) server_negotiation.cc:407] Connection header received
0809 22:49:49.825465 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:49.825470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:49.825566 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:49.825728 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:49.826203 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.826970 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.828783 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.829220 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.829352 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.829973 (+ 621us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:49.830549 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:49.830721 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:49.830822 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 22:49:49.830921 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:49:49.840605 19922 raft_consensus.cc:1261] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Refusing update from remote peer 78eb401e40f34f7eb81022428aef56d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:49.840581 20151 raft_consensus.cc:1261] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [term 1 FOLLOWER]: Refusing update from remote peer 78eb401e40f34f7eb81022428aef56d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:49.841715 20509 consensus_queue.cc:1035] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:49.842094 20507 consensus_queue.cc:1035] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:49.869849 19683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:49:49.870256 19683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:49:49.874753 19683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Sending DeleteTablet for 3 replicas of tablet 5ef7b15e417143d597bb160f00cc09d7
I20260809 22:49:49.876176 20124 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ef7b15e417143d597bb160f00cc09d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:49 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:40928
I20260809 22:49:49.876578 19893 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ef7b15e417143d597bb160f00cc09d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:49 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:38884
I20260809 22:49:49.877220 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 19788
I20260809 22:49:49.877310 20339 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ef7b15e417143d597bb160f00cc09d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:49:49 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:39072
I20260809 22:49:49.890426 20545 tablet_replica.cc:330] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: stopping tablet replica
I20260809 22:49:49.892100 20545 raft_consensus.cc:2229] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:49:49.894549 20542 tablet_replica.cc:330] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: stopping tablet replica
I20260809 22:49:49.894526 20545 raft_consensus.cc:2258] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:49.895095 20542 raft_consensus.cc:2229] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:49:49.895781 20542 raft_consensus.cc:2258] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:49.898456 20542 ts_tablet_manager.cc:1905] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:49.901170 20545 ts_tablet_manager.cc:1905] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:49:49.906050 20542 ts_tablet_manager.cc:1918] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:49.906303 20542 log.cc:1196] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/5ef7b15e417143d597bb160f00cc09d7
I20260809 22:49:49.906911 20542 ts_tablet_manager.cc:1939] T 5ef7b15e417143d597bb160f00cc09d7 P 13d1aa2ed32e4f07ba96dbd8113b8952: Deleting consensus metadata
I20260809 22:49:49.909195 20534 tablet_replica.cc:330] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: stopping tablet replica
I20260809 22:49:49.909231 19663 catalog_manager.cc:4841] TS 13d1aa2ed32e4f07ba96dbd8113b8952 (127.24.136.1:44493): tablet 5ef7b15e417143d597bb160f00cc09d7 (table test-table [id=824a538612d24995a2e85910059e5c9c]) successfully deleted
I20260809 22:49:49.909654 20534 raft_consensus.cc:2229] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:49:49.910132 20534 raft_consensus.cc:2258] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:49:49.912807 20534 ts_tablet_manager.cc:1905] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 22:49:49 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:49:49.916112 20545 ts_tablet_manager.cc:1918] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:49.916358 20545 log.cc:1196] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/5ef7b15e417143d597bb160f00cc09d7
I20260809 22:49:49.917006 20545 ts_tablet_manager.cc:1939] T 5ef7b15e417143d597bb160f00cc09d7 P 78eb401e40f34f7eb81022428aef56d4: Deleting consensus metadata
I20260809 22:49:49.919871 19659 catalog_manager.cc:4841] TS 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681): tablet 5ef7b15e417143d597bb160f00cc09d7 (table test-table [id=824a538612d24995a2e85910059e5c9c]) successfully deleted
I20260809 22:49:49.920549 20534 ts_tablet_manager.cc:1918] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:49:49.920742 20534 log.cc:1196] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/5ef7b15e417143d597bb160f00cc09d7
I20260809 22:49:49.921208 20534 ts_tablet_manager.cc:1939] T 5ef7b15e417143d597bb160f00cc09d7 P 3a5ee0af3c8d4da984658c7dc26a657c: Deleting consensus metadata
I20260809 22:49:49.923178 19660 catalog_manager.cc:4841] TS 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755): tablet 5ef7b15e417143d597bb160f00cc09d7 (table test-table [id=824a538612d24995a2e85910059e5c9c]) successfully deleted
I20260809 22:49:49.924602 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:49.878507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.24.136.1:44493)
0809 22:49:49.878703 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:49.878710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:49.888266 (+ 9556us) server_negotiation.cc:407] Connection header received
0809 22:49:49.889355 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:49.889364 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:49.889495 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:49.889605 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:49.895307 (+ 5702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.896747 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.906498 (+ 9751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:49.907312 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:49.907439 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:49.916183 (+ 8744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:49.916205 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:49.916227 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:49.916282 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:49.920400 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:49.921594 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:49.921606 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:49.921609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:49.921833 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:49.922431 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:49.922437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:49.922440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:49.922982 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:49.923279 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:49.923949 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 22:49:49.924208 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:49:49.925336 19993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:50.077880 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 19788
I20260809 22:49:50.104372 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 20014
Aug 09 22:49:50 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:49:50.123546 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:50.105406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40942 (local address 127.24.136.2:46755)
0809 22:49:50.105605 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:50.105610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:50.105633 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:49:50.105804 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:50.105808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:50.105884 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:50.105980 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:50.107848 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:50.108680 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:50.110682 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:50.111113 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:50.111224 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:50.116898 (+ 5674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:50.116914 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:50.116931 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:50.116983 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:50.120084 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:50.121173 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:50.121183 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:50.121186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:50.121356 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:50.121787 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:50.121790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:50.121792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:50.122230 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:50.122477 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:50.122972 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 22:49:50.123181 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:49:50.124326 20225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:50.273105 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 20014
I20260809 22:49:50.311705 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 20240
Aug 09 22:49:50 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:49:50.336259 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:50.312625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.24.136.3:43681)
0809 22:49:50.312801 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:50.312805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:50.312826 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:49:50.313061 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:50.313066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:50.313136 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:50.313246 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:50.315286 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:50.316242 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:50.318263 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:50.318758 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:50.318873 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:50.328728 (+ 9855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:50.328747 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:50.328765 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:50.328843 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:50.332721 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:50.333820 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:50.333832 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:50.333835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:50.334019 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:50.334565 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:50.334569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:50.334571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:50.334970 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:50.335239 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:50.335706 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 22:49:50.335928 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:49:50.337137 20452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:50.464537 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 20240
I20260809 22:49:50.488683 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 19635
I20260809 22:49:50.489850 19739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:50.602761 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 19635
I20260809 22:50:10.633589 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38989
--webserver_interface=127.24.136.62
--webserver_port=40915
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:50:11.002540 20553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:11.002888 20553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:11.003014 20553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:11.010866 20553 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 22:50:11.011008 20553 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 22:50:11.011049 20553 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 22:50:11.011132 20553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:50:11.011201 20553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:11.011286 20553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:11.011351 20553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:50:11.011422 20553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:50:11.022177 20553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=40915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:11.023695 20553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:11.025117 20553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:11.051283 20567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:11.051304 20562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:11.051983 20560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:11.053366 20553 server_base.cc:1034] running on GCE node
I20260809 22:50:11.054426 20553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:11.056339 20553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:11.057617 20553 hybrid_clock.cc:648] HybridClock initialized: now 1786315811057530 us; error 50 us; skew 500 ppm
I20260809 22:50:11.058094 20553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:11 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315811, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:11.062178 20553 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:50:11.064136 20553 webserver.cc:460] Webserver started at http://127.24.136.62:40915/ using document root <none> and password file <none>
I20260809 22:50:11.065004 20553 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:11.065129 20553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:11.072240 20553 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:50:11.075982 20582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:11.077667 20553 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:50:11.077894 20553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "14cf8de914614d39903c45fff959399a"
format_stamp: "Formatted at 2026-08-09 22:49:47 on dist-test-slave-pww0"
server_key: "b4aa4289808e2d584bd42f3d455b06b4"
server_key_iv: "f6c24ebef10d3f2fdb0ddca8f32feb31"
server_key_version: "encryptionkey@0"
I20260809 22:50:11.079073 20553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:11.112620 20553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:11.117616 20553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:11.117910 20553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:11.148049 20553 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38989
I20260809 22:50:11.148083 20672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38989 every 8 connection(s)
I20260809 22:50:11.150013 20553 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:50:11.156488 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 20553
I20260809 22:50:11.157433 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:44493
--local_ip_for_outbound_sockets=127.24.136.1
--tserver_master_addrs=127.24.136.62:38989
--webserver_port=34819
--webserver_interface=127.24.136.1
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:50:11.158532 20673 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:50:11.162456 20673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Bootstrap starting.
I20260809 22:50:11.187819 20673 log.cc:824] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:11.202004 20673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:50:11.202616 20673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Bootstrap complete.
I20260809 22:50:11.212503 20673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:50:11.213492 20673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14cf8de914614d39903c45fff959399a, State: Initialized, Role: FOLLOWER
I20260809 22:50:11.214221 20673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:50:11.214494 20673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:50:11.214629 20673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:50:11.214799 20673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:50:11.219158 20673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:50:11.219857 20673 leader_election.cc:302] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14cf8de914614d39903c45fff959399a; no voters:
I20260809 22:50:11.220551 20673 leader_election.cc:288] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:50:11.220719 20677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:50:11.221974 20677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 2 LEADER]: Becoming Leader. State: Replica: 14cf8de914614d39903c45fff959399a, State: Running, Role: LEADER
I20260809 22:50:11.222703 20677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:50:11.223127 20673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:50:11.225633 20680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14cf8de914614d39903c45fff959399a. Latest consensus state: current_term: 2 leader_uuid: "14cf8de914614d39903c45fff959399a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } } }
I20260809 22:50:11.226239 20680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: This master's current role is: LEADER
I20260809 22:50:11.226706 20679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "14cf8de914614d39903c45fff959399a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } } }
I20260809 22:50:11.227586 20679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: This master's current role is: LEADER
I20260809 22:50:11.240092 20689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:50:11.253885 20689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:50:11.255723 20689 catalog_manager.cc:1259] Loaded cluster ID: 87711c6231e64362b3a2b5211732697c
I20260809 22:50:11.255962 20689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:50:11.260987 20689 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:50:11.263113 20689 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Loaded TSK: 0
I20260809 22:50:11.305807 20689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Generated new TSK 1
I20260809 22:50:11.306483 20689 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:50:11.552400 20675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:11.552750 20675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:11.552894 20675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:11.560648 20675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:11.560797 20675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:11.560956 20675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:50:11.571569 20675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:44493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=34819
--tserver_master_addrs=127.24.136.62:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:11.573002 20675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:11.574414 20675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:11.591818 20720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:11.591912 20718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:11.592684 20722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:11.594029 20675 server_base.cc:1034] running on GCE node
I20260809 22:50:11.594785 20675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:11.595983 20675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:11.597249 20675 hybrid_clock.cc:648] HybridClock initialized: now 1786315811597194 us; error 41 us; skew 500 ppm
I20260809 22:50:11.597609 20675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:11 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315811, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:11.600999 20675 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:50:11.602705 20675 webserver.cc:460] Webserver started at http://127.24.136.1:34819/ using document root <none> and password file <none>
I20260809 22:50:11.603427 20675 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:11.603600 20675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:11.609702 20675 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:50:11.612843 20739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:11.614374 20675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:50:11.614558 20675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "13d1aa2ed32e4f07ba96dbd8113b8952"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "523e2fce95f9803eae39afd616b9aaae"
server_key_iv: "15dfd8dea66b648a6477c63278dd4902"
server_key_version: "encryptionkey@0"
I20260809 22:50:11.615676 20675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:11.643136 20675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:11.648042 20675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:11.648329 20675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:11.649437 20675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:11.650960 20675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:11.651060 20675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:11.651176 20675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:11.651265 20675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:11.686348 20675 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:44493
I20260809 22:50:11.686415 20942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:44493 every 8 connection(s)
I20260809 22:50:11.688020 20675 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:50:11.690882 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 20675
I20260809 22:50:11.691740 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:46755
--local_ip_for_outbound_sockets=127.24.136.2
--tserver_master_addrs=127.24.136.62:38989
--webserver_port=34511
--webserver_interface=127.24.136.2
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:50:11 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315811, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:50:11.714577 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:11.691289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:45619 (local address 127.24.136.62:38989)
0809 22:50:11.694293 (+ 3004us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:11.694310 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:11.694371 (+ 61us) server_negotiation.cc:407] Connection header received
0809 22:50:11.696049 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:11.696063 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:11.696414 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:11.696773 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:11.698422 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:11.699749 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:11.701355 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:11.702017 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:11.702202 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:11.706455 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:11.706464 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:11.706483 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:11.706532 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:11.709551 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:11.710640 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:11.710647 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:11.710653 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:11.710841 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:11.711428 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:11.711431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:11.711433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:11.712030 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:11.712197 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:11.713199 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0809 22:50:11.713557 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2756,"thread_start_us":2630,"threads_started":1}
I20260809 22:50:11.714999 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:11.691502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:45619)
0809 22:50:11.693345 (+ 1843us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:11.693402 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:11.695422 (+ 2020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:11.696971 (+ 1549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:11.696984 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:11.697626 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:11.698177 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:11.698195 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:11.699936 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:11.699940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:11.701191 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:11.701205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:11.702275 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:11.702284 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:11.702562 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:11.703238 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:50:11.703271 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:11.706068 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:11.709754 (+ 3686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:11.709766 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:11.709786 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:11.710497 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:11.710982 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:11.710987 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:11.710992 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:11.711318 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:11.712195 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:11.712205 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:11.712697 (+ 492us) client_negotiation.cc:769] Sending connection context
0809 22:50:11.713072 (+ 375us) client_negotiation.cc:240] Negotiation successful
0809 22:50:11.713639 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1497,"thread_start_us":1247,"threads_started":1}
I20260809 22:50:11.718026 20944 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38989
I20260809 22:50:11.718354 20944 heartbeater.cc:464] Registering TS with master...
I20260809 22:50:11.719131 20944 heartbeater.cc:511] Master 127.24.136.62:38989 requested a full tablet report, sending...
I20260809 22:50:11.721438 20610 ts_manager.cc:194] Registered new tserver with Master: 13d1aa2ed32e4f07ba96dbd8113b8952 (127.24.136.1:44493)
I20260809 22:50:11.722993 20610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:45619
W20260809 22:50:12.062853 20949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:12.063212 20949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:12.063356 20949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:12.070999 20949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:12.071137 20949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:12.071261 20949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:50:12.082074 20949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:46755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=34511
--tserver_master_addrs=127.24.136.62:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:12.083549 20949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:12.084996 20949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:12.099071 20966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:12.099005 20964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:12.099050 20963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:12.100062 20949 server_base.cc:1034] running on GCE node
I20260809 22:50:12.100746 20949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:12.101799 20949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:12.103030 20949 hybrid_clock.cc:648] HybridClock initialized: now 1786315812102976 us; error 39 us; skew 500 ppm
I20260809 22:50:12.103394 20949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:12 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315812, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:12.107056 20949 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:50:12.108834 20949 webserver.cc:460] Webserver started at http://127.24.136.2:34511/ using document root <none> and password file <none>
I20260809 22:50:12.109608 20949 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:12.109772 20949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:12.115927 20949 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:50:12.119074 20986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:12.120817 20949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:50:12.121008 20949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "3a5ee0af3c8d4da984658c7dc26a657c"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "bda3acbe6e6ada2c9cefa5863d989436"
server_key_iv: "7bb0e993b1b7899d8876b4484ed39e60"
server_key_version: "encryptionkey@0"
I20260809 22:50:12.122093 20949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:12.144117 20949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:12.148895 20949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:12.149174 20949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:12.150319 20949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:12.151883 20949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:12.151989 20949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:12.152104 20949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:12.152176 20949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:12.192461 20949 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:46755
I20260809 22:50:12.192515 21184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:46755 every 8 connection(s)
I20260809 22:50:12.194043 20949 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:50:12.195920 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 20949
I20260809 22:50:12.196732 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:43681
--local_ip_for_outbound_sockets=127.24.136.3
--tserver_master_addrs=127.24.136.62:38989
--webserver_port=40319
--webserver_interface=127.24.136.3
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:50:12 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315812, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:50:12.216214 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.196802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:49129 (local address 127.24.136.62:38989)
0809 22:50:12.197011 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.197017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.198449 (+ 1432us) server_negotiation.cc:407] Connection header received
0809 22:50:12.200440 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.200447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.200576 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.200673 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:12.202387 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.203114 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.204842 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.205303 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.205432 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.209788 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:12.209794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:12.209798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:12.209838 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:12.212545 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.213544 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.213548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.213550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.213677 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.214162 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.214165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.214167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.214485 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:12.214603 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.215665 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.215934 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:50:12.217605 21188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.197446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:49129)
0809 22:50:12.198266 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:12.198314 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:12.200118 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:12.200971 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:12.200981 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:12.201543 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:12.202181 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.202201 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.203297 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.203301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.204669 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.204685 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.205620 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.205629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.205908 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.206636 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:50:12.206667 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:12.209450 (+ 2783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:12.212734 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:12.212745 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:12.212774 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:12.213408 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:12.213791 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:12.213796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:12.213799 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:12.214064 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:12.214625 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:12.214634 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:12.215123 (+ 489us) client_negotiation.cc:769] Sending connection context
0809 22:50:12.215510 (+ 387us) client_negotiation.cc:240] Negotiation successful
0809 22:50:12.216123 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":347,"threads_started":1}
I20260809 22:50:12.219355 21185 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38989
I20260809 22:50:12.219719 21185 heartbeater.cc:464] Registering TS with master...
I20260809 22:50:12.220499 21185 heartbeater.cc:511] Master 127.24.136.62:38989 requested a full tablet report, sending...
I20260809 22:50:12.222158 20610 ts_manager.cc:194] Registered new tserver with Master: 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
I20260809 22:50:12.223042 20610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:49129
W20260809 22:50:12.559581 21190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:12.559931 21190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:12.560081 21190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:12.567816 21190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:12.567972 21190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:12.568135 21190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:50:12.578289 21190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:43681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=40319
--tserver_master_addrs=127.24.136.62:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:12.579751 21190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:12.581156 21190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:12.595914 21198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:12.595978 21197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:12.595955 21203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:12.601382 21190 server_base.cc:1034] running on GCE node
I20260809 22:50:12.602151 21190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:12.603214 21190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:12.604478 21190 hybrid_clock.cc:648] HybridClock initialized: now 1786315812604404 us; error 58 us; skew 500 ppm
I20260809 22:50:12.604883 21190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:12 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315812, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:12.608731 21190 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:50:12.610509 21190 webserver.cc:460] Webserver started at http://127.24.136.3:40319/ using document root <none> and password file <none>
I20260809 22:50:12.611285 21190 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:12.611430 21190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:12.617401 21190 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:50:12.620519 21213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:12.622027 21190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:50:12.622200 21190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "78eb401e40f34f7eb81022428aef56d4"
format_stamp: "Formatted at 2026-08-09 22:49:49 on dist-test-slave-pww0"
server_key: "d199bb72824a7ab30b6b6e952d2df832"
server_key_iv: "937679f8720373dbd2302238fd6bb6d5"
server_key_version: "encryptionkey@0"
I20260809 22:50:12.623211 21190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:12.645263 21190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:12.649907 21190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:12.650193 21190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:12.651259 21190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:12.652752 21190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:12.652853 21190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:12.652964 21190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:12.653031 21190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:12.687076 21190 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:43681
I20260809 22:50:12.687120 21410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:43681 every 8 connection(s)
I20260809 22:50:12.689177 21190 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:50:12.692159 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 21190
Aug 09 22:50:12 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315812, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:50:12.709138 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.693184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.24.136.62:38989)
0809 22:50:12.693612 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.693618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.694065 (+ 447us) server_negotiation.cc:407] Connection header received
0809 22:50:12.694264 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.694271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.694363 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.694491 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:12.696834 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.697993 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.699529 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.700019 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.700152 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.703377 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:12.703382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:12.703385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:12.703421 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:12.706161 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.706915 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.706919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.706921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.707062 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.707501 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.707507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.707510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.707827 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:12.708099 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.708462 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.708742 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":223,"threads_started":1}
I20260809 22:50:12.712769 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.692335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:35149 (local address 127.24.136.62:38989)
0809 22:50:12.692546 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.692552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.694379 (+ 1827us) server_negotiation.cc:407] Connection header received
0809 22:50:12.697013 (+ 2634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.697018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.697106 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.697205 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:12.699107 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.699963 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.701828 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.702261 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.702390 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.706368 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:12.706373 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:12.706376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:12.706410 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:12.709115 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.710072 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.710077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.710079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.710201 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.710718 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.710722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.710724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.711098 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:12.711252 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.712243 (+ 991us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.712449 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:50:12.713788 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.693131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:35149)
0809 22:50:12.694220 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:12.694271 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:12.696732 (+ 2461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:12.697524 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:12.697538 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:12.698301 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:12.698921 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.698942 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.700164 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.700168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.701662 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.701677 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.702519 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.702528 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.702794 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.703518 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:50:12.703550 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:12.706145 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:12.709258 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:12.709267 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:12.709290 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:12.709958 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:12.710328 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:12.710333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:12.710337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:12.710590 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:12.711269 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:12.711276 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:12.711765 (+ 489us) client_negotiation.cc:769] Sending connection context
0809 22:50:12.712113 (+ 348us) client_negotiation.cc:240] Negotiation successful
0809 22:50:12.712592 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":547,"threads_started":1}
I20260809 22:50:12.715313 21411 heartbeater.cc:347] Connected to a master server at 127.24.136.62:38989
I20260809 22:50:12.715660 21411 heartbeater.cc:464] Registering TS with master...
I20260809 22:50:12.716410 21411 heartbeater.cc:511] Master 127.24.136.62:38989 requested a full tablet report, sending...
I20260809 22:50:12.717944 20610 ts_manager.cc:194] Registered new tserver with Master: 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681)
I20260809 22:50:12.718845 20610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:35149
I20260809 22:50:12.724645 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:50:12.727166 20944 heartbeater.cc:503] Master 127.24.136.62:38989 was elected leader, sending a full tablet report...
I20260809 22:50:12.732241 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.725679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.24.136.62:38989)
0809 22:50:12.725870 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.725876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.726354 (+ 478us) server_negotiation.cc:407] Connection header received
0809 22:50:12.726533 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.726540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.726772 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.726898 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:50:12.727433 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.728609 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.730157 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.730594 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.730692 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.731273 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:50:12.731689 (+ 416us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 22:50:12.732025 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48464: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 22:50:12.732362 20947 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48464: token expired
W20260809 22:50:12.733021 20471 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.136.62:38989)
I20260809 22:50:12.746054 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.733630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.24.136.62:38989)
0809 22:50:12.733802 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.733806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.733824 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:50:12.733974 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.733979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.734061 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.734144 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:12.735548 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.736334 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.737651 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.738067 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.738161 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.740829 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:12.740834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:12.740838 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:12.740877 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:12.743650 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.744353 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.744357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.744360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.744492 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.744814 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.744817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.744818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.745161 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:12.745361 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.745584 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.745787 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:50:12.749748 20610 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:50:12.757448 20610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:50:12.793937 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.770837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:57410)
0809 22:50:12.784323 (+ 13486us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:12.784378 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:12.784668 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:12.786056 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:12.786064 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:12.786098 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:12.786524 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.786540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.788389 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.788395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.790762 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.790780 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.792806 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.792812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.793020 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.793079 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 22:50:12.793237 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 22:50:12.793455 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13341,"mutex_wait_us":1,"spinlock_wait_cycles":6912,"thread_start_us":13178,"threads_started":1}
I20260809 22:50:12.794819 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.771294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57410 (local address 127.24.136.1:44493)
0809 22:50:12.781868 (+ 10574us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.781880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.785031 (+ 3151us) server_negotiation.cc:407] Connection header received
0809 22:50:12.785161 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.785170 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.785486 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.785719 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:12.786726 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.788262 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.790978 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.792553 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.792681 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.792918 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.794244 (+ 1326us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.794452 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10369,"thread_start_us":10187,"threads_started":1}
I20260809 22:50:12.795437 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.770600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:43681 (local address 127.0.0.1:55522)
0809 22:50:12.783431 (+ 12831us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:12.784375 (+ 944us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:12.784668 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:12.787437 (+ 2769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:12.787442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:12.787497 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:12.787824 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.787837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.790050 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.790056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.792015 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.792030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.794538 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.794545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.794713 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.794747 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:50:12.794857 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 22:50:12.795004 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12461,"mutex_wait_us":10,"spinlock_wait_cycles":23552,"thread_start_us":12261,"threads_started":1}
I20260809 22:50:12.795513 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.770775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55522 (local address 127.24.136.3:43681)
0809 22:50:12.786985 (+ 16210us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.786992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.787021 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:50:12.787097 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.787102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.787271 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.787442 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:12.788014 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.789897 (+ 1883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.792211 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.794314 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.794449 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.794687 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.794952 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.795127 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16049,"thread_start_us":15887,"threads_started":1}
I20260809 22:50:12.796317 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.771573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45488 (local address 127.24.136.2:46755)
0809 22:50:12.784686 (+ 13113us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.784695 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.785970 (+ 1275us) server_negotiation.cc:407] Connection header received
0809 22:50:12.786091 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.786099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.786366 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.786582 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:12.787436 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.789829 (+ 2393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.792516 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.793984 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.794473 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.794690 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.795813 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.795998 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12938,"thread_start_us":12821,"threads_started":1}
I20260809 22:50:12.795687 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.771042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:46755 (local address 127.0.0.1:45488)
0809 22:50:12.785690 (+ 14648us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:12.785710 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:12.785831 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:12.786838 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:12.786842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:12.786868 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:12.787212 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.787225 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.790034 (+ 2809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.790040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.792308 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.792325 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.794837 (+ 2512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.794844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.795024 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.795055 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:50:12.795149 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 22:50:12.795293 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14487,"mutex_wait_us":99,"thread_start_us":13880,"threads_started":1}
I20260809 22:50:12.799177 20815 tablet_service.cc:1467] Processing CreateTablet for tablet ff6f1bb5eb2043c19a4375b7d803831f (DEFAULT_TABLE table=test-table [id=fc456353bd8d456bb726756370ecb6ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:50:12.799177 21288 tablet_service.cc:1467] Processing CreateTablet for tablet ff6f1bb5eb2043c19a4375b7d803831f (DEFAULT_TABLE table=test-table [id=fc456353bd8d456bb726756370ecb6ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:50:12.800523 21288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6f1bb5eb2043c19a4375b7d803831f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:12.800628 20815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6f1bb5eb2043c19a4375b7d803831f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:12.802771 21063 tablet_service.cc:1467] Processing CreateTablet for tablet ff6f1bb5eb2043c19a4375b7d803831f (DEFAULT_TABLE table=test-table [id=fc456353bd8d456bb726756370ecb6ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:50:12.804435 21063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6f1bb5eb2043c19a4375b7d803831f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:12.813383 21427 tablet_bootstrap.cc:492] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: Bootstrap starting.
I20260809 22:50:12.816423 21428 tablet_bootstrap.cc:492] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Bootstrap starting.
I20260809 22:50:12.818723 21427 tablet_bootstrap.cc:654] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:12.819831 21429 tablet_bootstrap.cc:492] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: Bootstrap starting.
I20260809 22:50:12.820209 21428 tablet_bootstrap.cc:654] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:12.820192 21427 log.cc:824] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:12.821364 21428 log.cc:824] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:12.823539 21427 tablet_bootstrap.cc:492] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: No bootstrap required, opened a new log
I20260809 22:50:12.823699 21429 tablet_bootstrap.cc:654] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:12.823848 21427 ts_tablet_manager.cc:1397] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260809 22:50:12.824504 21428 tablet_bootstrap.cc:492] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: No bootstrap required, opened a new log
I20260809 22:50:12.824772 21428 ts_tablet_manager.cc:1397] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:50:12.824766 21429 log.cc:824] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:12.834173 21429 tablet_bootstrap.cc:492] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: No bootstrap required, opened a new log
I20260809 22:50:12.834479 21429 ts_tablet_manager.cc:1397] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260809 22:50:12.834923 21427 raft_consensus.cc:348] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:12.835147 21428 raft_consensus.cc:348] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:12.835440 21427 raft_consensus.cc:374] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:12.835575 21427 raft_consensus.cc:729] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78eb401e40f34f7eb81022428aef56d4, State: Initialized, Role: FOLLOWER
I20260809 22:50:12.835613 21428 raft_consensus.cc:374] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:12.835698 21428 raft_consensus.cc:729] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13d1aa2ed32e4f07ba96dbd8113b8952, State: Initialized, Role: FOLLOWER
I20260809 22:50:12.836299 21428 consensus_queue.cc:260] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:12.836298 21427 consensus_queue.cc:260] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:12.840770 21411 heartbeater.cc:503] Master 127.24.136.62:38989 was elected leader, sending a full tablet report...
I20260809 22:50:12.841125 21428 ts_tablet_manager.cc:1428] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Time spent starting tablet: real 0.016s user 0.009s sys 0.008s
I20260809 22:50:12.841369 21427 ts_tablet_manager.cc:1428] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: Time spent starting tablet: real 0.017s user 0.009s sys 0.007s
I20260809 22:50:12.844784 21429 raft_consensus.cc:348] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:12.845302 21429 raft_consensus.cc:374] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:12.845418 21429 raft_consensus.cc:729] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5ee0af3c8d4da984658c7dc26a657c, State: Initialized, Role: FOLLOWER
I20260809 22:50:12.846050 21429 consensus_queue.cc:260] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:12.847275 21185 heartbeater.cc:503] Master 127.24.136.62:38989 was elected leader, sending a full tablet report...
I20260809 22:50:12.848062 21429 ts_tablet_manager.cc:1428] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: Time spent starting tablet: real 0.013s user 0.012s sys 0.002s
I20260809 22:50:12.869362 21434 raft_consensus.cc:482] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:50:12.869645 21434 raft_consensus.cc:504] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:12.871361 21434 leader_election.cc:288] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681), 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
I20260809 22:50:12.878273 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.871985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:33209 (local address 127.24.136.2:46755)
0809 22:50:12.872261 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.872268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.872773 (+ 505us) server_negotiation.cc:407] Connection header received
0809 22:50:12.872968 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.872973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.873040 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.873119 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:12.873722 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.874541 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.876544 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.877337 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.877485 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.877564 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.877886 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.878021 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260809 22:50:12.878336 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.871852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:36589 (local address 127.24.136.3:43681)
0809 22:50:12.872061 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.872067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.872946 (+ 879us) server_negotiation.cc:407] Connection header received
0809 22:50:12.873111 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.873115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.873172 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.873244 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:12.873853 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.874601 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.876564 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.877369 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.877520 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.877598 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.877953 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.878080 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:50:12.878558 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.871768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:46755 (local address 127.24.136.1:33209)
0809 22:50:12.872656 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:12.872672 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:12.872836 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:12.873151 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:12.873156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:12.873181 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:12.873568 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.873578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.874669 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.874675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.876383 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.876399 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.877559 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.877565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.877734 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.877758 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:50:12.877843 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 22:50:12.877977 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":54,"thread_start_us":505,"threads_started":1}
I20260809 22:50:12.878810 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.871814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:43681 (local address 127.24.136.1:36589)
0809 22:50:12.872827 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:12.872846 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:12.872980 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:12.873276 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:12.873281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:12.873326 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:12.873693 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.873702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.874733 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.874739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.876410 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:12.876423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.877596 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.877603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:12.877797 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.877821 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:50:12.877907 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:50:12.878037 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":33,"thread_start_us":684,"threads_started":1}
I20260809 22:50:12.879592 21095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6f1bb5eb2043c19a4375b7d803831f" candidate_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" is_pre_election: true
I20260809 22:50:12.879799 21319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6f1bb5eb2043c19a4375b7d803831f" candidate_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78eb401e40f34f7eb81022428aef56d4" is_pre_election: true
I20260809 22:50:12.880127 21095 raft_consensus.cc:2454] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13d1aa2ed32e4f07ba96dbd8113b8952 in term 0.
I20260809 22:50:12.880278 21319 raft_consensus.cc:2454] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13d1aa2ed32e4f07ba96dbd8113b8952 in term 0.
I20260809 22:50:12.880811 20741 leader_election.cc:302] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13d1aa2ed32e4f07ba96dbd8113b8952, 3a5ee0af3c8d4da984658c7dc26a657c; no voters:
I20260809 22:50:12.881222 21434 raft_consensus.cc:2781] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:50:12.881381 21434 raft_consensus.cc:482] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:50:12.881482 21434 raft_consensus.cc:3037] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:12.883836 21434 raft_consensus.cc:504] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:12.884513 21434 leader_election.cc:288] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 1 election: Requested vote from peers 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681), 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
I20260809 22:50:12.885078 21319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6f1bb5eb2043c19a4375b7d803831f" candidate_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78eb401e40f34f7eb81022428aef56d4"
I20260809 22:50:12.885087 21095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6f1bb5eb2043c19a4375b7d803831f" candidate_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5ee0af3c8d4da984658c7dc26a657c"
I20260809 22:50:12.885273 21319 raft_consensus.cc:3037] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:12.885283 21095 raft_consensus.cc:3037] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:12.887929 21319 raft_consensus.cc:2454] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13d1aa2ed32e4f07ba96dbd8113b8952 in term 1.
I20260809 22:50:12.888015 21095 raft_consensus.cc:2454] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13d1aa2ed32e4f07ba96dbd8113b8952 in term 1.
I20260809 22:50:12.888407 20740 leader_election.cc:302] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13d1aa2ed32e4f07ba96dbd8113b8952, 78eb401e40f34f7eb81022428aef56d4; no voters:
I20260809 22:50:12.888747 21434 raft_consensus.cc:2781] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:50:12.889267 21434 raft_consensus.cc:686] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 LEADER]: Becoming Leader. State: Replica: 13d1aa2ed32e4f07ba96dbd8113b8952, State: Running, Role: LEADER
I20260809 22:50:12.889744 21434 consensus_queue.cc:237] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:12.894446 20609 catalog_manager.cc:5495] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13d1aa2ed32e4f07ba96dbd8113b8952 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } health_report { overall_health: UNKNOWN } } }
I20260809 22:50:12.924376 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.918896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57414 (local address 127.24.136.1:44493)
0809 22:50:12.919106 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.919111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.919130 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:50:12.919280 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.919284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.919380 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.919539 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:50:12.919969 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.920682 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.922072 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.922524 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.922623 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.923112 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:50:12.923790 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:50:12.923904 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.924024 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.924102 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:50:12.933241 21095 raft_consensus.cc:1261] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [term 1 FOLLOWER]: Refusing update from remote peer 13d1aa2ed32e4f07ba96dbd8113b8952: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:50:12.933257 21319 raft_consensus.cc:1261] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [term 1 FOLLOWER]: Refusing update from remote peer 13d1aa2ed32e4f07ba96dbd8113b8952: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:50:12.934274 21442 consensus_queue.cc:1035] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:50:12.934648 21434 consensus_queue.cc:1035] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:50:12.940706 20945 tablet.cc:2369] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:50:12.941793 21412 tablet.cc:2369] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:50:12.946835 21187 tablet.cc:2369] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:50:12.948017 21448 mvcc.cc:205] Tried to move back new op lower bound from 7316749569763835904 to 7316749569600131072. Current Snapshot: MvccSnapshot[applied={T|T < 7316749569763835904}]
I20260809 22:50:12.962739 20609 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:50:12.963032 20609 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:50:12.967084 20609 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Sending DeleteTablet for 3 replicas of tablet ff6f1bb5eb2043c19a4375b7d803831f
I20260809 22:50:12.968513 21288 tablet_service.cc:1514] Processing DeleteTablet for tablet ff6f1bb5eb2043c19a4375b7d803831f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:50:12 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:55522
I20260809 22:50:12.968524 20815 tablet_service.cc:1514] Processing DeleteTablet for tablet ff6f1bb5eb2043c19a4375b7d803831f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:50:12 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:57410
I20260809 22:50:12.968801 21063 tablet_service.cc:1514] Processing DeleteTablet for tablet ff6f1bb5eb2043c19a4375b7d803831f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:50:12 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:45488
I20260809 22:50:12.969898 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 20675
I20260809 22:50:12.991490 21468 tablet_replica.cc:330] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: stopping tablet replica
I20260809 22:50:12.992067 21463 tablet_replica.cc:330] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: stopping tablet replica
I20260809 22:50:12.992530 21463 raft_consensus.cc:2229] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:50:12.992581 21468 raft_consensus.cc:2229] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:50:12.993117 21463 raft_consensus.cc:2258] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:50:12.993335 21468 raft_consensus.cc:2258] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:50:12.994273 21466 tablet_replica.cc:330] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: stopping tablet replica
I20260809 22:50:12.994724 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:12.971633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57416 (local address 127.24.136.1:44493)
0809 22:50:12.971816 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:12.971821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:12.971841 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:50:12.971902 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:12.971908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:12.971985 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:12.972099 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:12.974307 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.975632 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.977732 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:12.978467 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:12.978594 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:12.983782 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:12.983807 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:12.983831 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:12.983903 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:12.989610 (+ 5707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.990972 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.990987 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.990991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.991227 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:12.991812 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:12.991818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:12.991821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:12.992348 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:12.992741 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:12.994079 (+ 1338us) server_negotiation.cc:299] Negotiation successful
0809 22:50:12.994348 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:50:12.994763 21466 raft_consensus.cc:2229] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:50:12.995764 21466 raft_consensus.cc:2258] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:50:12.995929 21468 ts_tablet_manager.cc:1905] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:50:12.996690 20932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:12.998206 21463 ts_tablet_manager.cc:1905] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:50:13.002589 21468 ts_tablet_manager.cc:1918] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:50:13.002767 21468 log.cc:1196] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/ff6f1bb5eb2043c19a4375b7d803831f
I20260809 22:50:13.003221 21468 ts_tablet_manager.cc:1939] T ff6f1bb5eb2043c19a4375b7d803831f P 3a5ee0af3c8d4da984658c7dc26a657c: Deleting consensus metadata
I20260809 22:50:13.004977 20586 catalog_manager.cc:4841] TS 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755): tablet ff6f1bb5eb2043c19a4375b7d803831f (table test-table [id=fc456353bd8d456bb726756370ecb6ad]) successfully deleted
I20260809 22:50:13.006290 21463 ts_tablet_manager.cc:1918] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:50:13.006484 21463 log.cc:1196] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/ff6f1bb5eb2043c19a4375b7d803831f
I20260809 22:50:13.007040 21463 ts_tablet_manager.cc:1939] T ff6f1bb5eb2043c19a4375b7d803831f P 78eb401e40f34f7eb81022428aef56d4: Deleting consensus metadata
I20260809 22:50:13.009143 20584 catalog_manager.cc:4841] TS 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681): tablet ff6f1bb5eb2043c19a4375b7d803831f (table test-table [id=fc456353bd8d456bb726756370ecb6ad]) successfully deleted
I20260809 22:50:13.098366 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 20675
W20260809 22:50:13.124964 20589 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.136.1:44493 (error 108)
I20260809 22:50:13.125276 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 20949
I20260809 22:50:13.131963 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.131369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:57426)
0809 22:50:13.131608 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:13.131747 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 22:50:13.132236 20589 catalog_manager.cc:4584] TS 13d1aa2ed32e4f07ba96dbd8113b8952 (127.24.136.1:44493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff6f1bb5eb2043c19a4375b7d803831f: Network error: Client connection negotiation failed: client connection to 127.24.136.1:44493: connect: Connection refused (error 111)
I20260809 22:50:13.142036 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.126575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.24.136.2:46755)
0809 22:50:13.126769 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:13.126775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:13.126798 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:50:13.126873 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:13.126879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:13.126966 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:13.127101 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:13.129888 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:13.130940 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:13.132712 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:13.133130 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:13.133251 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:13.136018 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:13.136047 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:13.136063 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:13.136106 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:13.138903 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:13.139871 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:13.139879 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:13.139882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:13.140019 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:13.140425 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:13.140429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:13.140431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:13.140849 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:13.141154 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:13.141474 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 22:50:13.141699 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:50:13.142789 21177 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:13.185997 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.185464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:57442)
0809 22:50:13.185698 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:13.185815 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 22:50:13.187117 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.186713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:57454)
0809 22:50:13.186897 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:13.186978 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 22:50:13.233302 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 20949
I20260809 22:50:13.247658 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.247052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:57456)
0809 22:50:13.247290 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:13.247386 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 22:50:13.248836 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.248472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:57458)
0809 22:50:13.248630 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:13.248712 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:50:13.260347 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 21190
I20260809 22:50:13.275308 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.261152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.24.136.3:43681)
0809 22:50:13.261350 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:13.261357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:13.261379 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:50:13.261605 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:13.261609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:13.261705 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:13.261798 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:13.263351 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:13.264153 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:13.265710 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:13.266160 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:13.266289 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:13.269184 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:13.269200 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:13.269216 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:13.269258 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:13.272224 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:13.273023 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:13.273032 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:13.273034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:13.273209 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:13.273619 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:13.273624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:13.273627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:13.274006 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:13.274310 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:13.274708 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 22:50:13.274966 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:50:13.276136 21400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:13.329602 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.329068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:57464)
0809 22:50:13.329323 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:13.329431 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 22:50:13.330736 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.330332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:57470)
0809 22:50:13.330539 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:13.330616 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 22:50:13.366539 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 21190
I20260809 22:50:13.390539 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 20553
I20260809 22:50:13.391834 20665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:13.510145 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:13.509321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:44493 (local address 127.0.0.1:57484)
0809 22:50:13.509521 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:13.509648 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:44493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 22:50:13.510512 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 20553
I20260809 22:50:33.549958 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38989
--webserver_interface=127.24.136.62
--webserver_port=40915
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:50:33.910094 21476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:33.910434 21476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:33.910575 21476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:33.918087 21476 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 22:50:33.918212 21476 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 22:50:33.918279 21476 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 22:50:33.918352 21476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:50:33.918426 21476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:33.918525 21476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:33.918586 21476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:50:33.918661 21476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:50:33.929190 21476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=40915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:33.930748 21476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:33.932268 21476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:33.944916 21491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:33.944873 21487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:33.944965 21485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:33.945556 21476 server_base.cc:1034] running on GCE node
I20260809 22:50:33.946571 21476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:33.948169 21476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:33.949420 21476 hybrid_clock.cc:648] HybridClock initialized: now 1786315833949349 us; error 39 us; skew 500 ppm
I20260809 22:50:33.949795 21476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:33 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315833, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:33.957667 21476 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:50:33.959540 21476 webserver.cc:460] Webserver started at http://127.24.136.62:40915/ using document root <none> and password file <none>
I20260809 22:50:33.960366 21476 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:33.960491 21476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:33.966828 21476 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 22:50:33.970084 21509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:33.971623 21476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:50:33.971837 21476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "14cf8de914614d39903c45fff959399a"
format_stamp: "Formatted at 2026-08-09 22:49:47 on dist-test-slave-pww0"
server_key: "b4aa4289808e2d584bd42f3d455b06b4"
server_key_iv: "f6c24ebef10d3f2fdb0ddca8f32feb31"
server_key_version: "encryptionkey@0"
I20260809 22:50:33.972875 21476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:33.991940 21476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:33.997058 21476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:33.997344 21476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:34.015316 21476 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38989
I20260809 22:50:34.015339 21609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38989 every 8 connection(s)
I20260809 22:50:34.017028 21476 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:50:34.022883 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 21476
I20260809 22:50:34.023819 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:44493
--local_ip_for_outbound_sockets=127.24.136.1
--tserver_master_addrs=127.24.136.62:38989
--webserver_port=34819
--webserver_interface=127.24.136.1
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:50:34.024927 21610 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:50:34.028679 21610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Bootstrap starting.
I20260809 22:50:34.050688 21610 log.cc:824] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:34.068941 21610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:50:34.069543 21610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Bootstrap complete.
I20260809 22:50:34.079355 21610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:50:34.080399 21610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14cf8de914614d39903c45fff959399a, State: Initialized, Role: FOLLOWER
I20260809 22:50:34.081079 21610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:50:34.081341 21610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:50:34.081465 21610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:50:34.081645 21610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:50:34.085981 21610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:50:34.086544 21610 leader_election.cc:302] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14cf8de914614d39903c45fff959399a; no voters:
I20260809 22:50:34.087272 21610 leader_election.cc:288] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 22:50:34.087502 21617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 22:50:34.088537 21617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [term 3 LEADER]: Becoming Leader. State: Replica: 14cf8de914614d39903c45fff959399a, State: Running, Role: LEADER
I20260809 22:50:34.089210 21617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } }
I20260809 22:50:34.089411 21610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:50:34.092386 21620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14cf8de914614d39903c45fff959399a. Latest consensus state: current_term: 3 leader_uuid: "14cf8de914614d39903c45fff959399a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } } }
I20260809 22:50:34.092051 21619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "14cf8de914614d39903c45fff959399a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14cf8de914614d39903c45fff959399a" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38989 } } }
I20260809 22:50:34.093207 21619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: This master's current role is: LEADER
I20260809 22:50:34.099319 21620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a [sys.catalog]: This master's current role is: LEADER
I20260809 22:50:34.100999 21626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:50:34.111719 21626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:50:34.113708 21626 catalog_manager.cc:1259] Loaded cluster ID: 87711c6231e64362b3a2b5211732697c
I20260809 22:50:34.113931 21626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:50:34.119133 21626 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:50:34.121415 21626 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Loaded TSK: 0
I20260809 22:50:34.121762 21626 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 14cf8de914614d39903c45fff959399a: Loaded TSK: 1
I20260809 22:50:34.123291 21646 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
F20260809 22:50:34.123142 21626 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f55242ea9ad google::LogMessage::Fail() at ??:0
@ 0x7f55242ed88a google::LogMessage::SendToLog() at ??:0
@ 0x7f55242ea4de google::LogMessage::Flush() at ??:0
@ 0x7f55242eba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f552804e7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f552803d73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f5539d191c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f5539d18783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f5539da42a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f5539da40c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f5539dee5e9 std::function<>::operator()() at ??:0
@ 0x7f5539d1ccdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5539d1ac36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5539da2610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f5539da2439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5526f36345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5526f3991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f5526f39799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5526f14eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f552a1ee6db start_thread at ??:0
@ 0x7f552318461f clone at ??:0
W20260809 22:50:34.429348 21614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:34.429692 21614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:34.429844 21614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:34.437841 21614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:34.438017 21614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:34.438202 21614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:50:34.449014 21614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:44493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=34819
--tserver_master_addrs=127.24.136.62:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:34.450476 21614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:34.451977 21614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:34.466737 21706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:34.466830 21703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:34.466867 21702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:34.467599 21614 server_base.cc:1034] running on GCE node
I20260809 22:50:34.468329 21614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:34.469437 21614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:34.470672 21614 hybrid_clock.cc:648] HybridClock initialized: now 1786315834470613 us; error 56 us; skew 500 ppm
I20260809 22:50:34.471030 21614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:34 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315834, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:34.474689 21614 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:50:34.476426 21614 webserver.cc:460] Webserver started at http://127.24.136.1:34819/ using document root <none> and password file <none>
I20260809 22:50:34.477185 21614 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:34.477324 21614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:34.483258 21614 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:50:34.486562 21715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:34.488260 21614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:50:34.488442 21614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "13d1aa2ed32e4f07ba96dbd8113b8952"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "523e2fce95f9803eae39afd616b9aaae"
server_key_iv: "15dfd8dea66b648a6477c63278dd4902"
server_key_version: "encryptionkey@0"
I20260809 22:50:34.489470 21614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:34.525313 21614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:34.530027 21614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:34.530310 21614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:34.531361 21614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:34.533345 21727 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 22:50:34.539274 21614 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 22:50:34.539420 21614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 22:50:34.539589 21614 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 22:50:34.542958 21614 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 22:50:34.543044 21614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260809 22:50:34.543437 21727 tablet_bootstrap.cc:492] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Bootstrap starting.
I20260809 22:50:34.552203 21727 log.cc:824] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:34.567831 21727 tablet_bootstrap.cc:492] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:50:34.568754 21727 tablet_bootstrap.cc:492] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Bootstrap complete.
I20260809 22:50:34.569295 21727 ts_tablet_manager.cc:1397] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Time spent bootstrapping tablet: real 0.026s user 0.015s sys 0.008s
I20260809 22:50:34.578964 21727 raft_consensus.cc:348] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:34.580718 21727 raft_consensus.cc:729] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13d1aa2ed32e4f07ba96dbd8113b8952, State: Initialized, Role: FOLLOWER
I20260809 22:50:34.581614 21727 consensus_queue.cc:260] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:34.583305 21727 ts_tablet_manager.cc:1428] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952: Time spent starting tablet: real 0.014s user 0.002s sys 0.012s
I20260809 22:50:34.587225 21614 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:44493
I20260809 22:50:34.587262 21900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:44493 every 8 connection(s)
I20260809 22:50:34.588575 21614 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:50:34.593873 21908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:34.591600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:59545)
0809 22:50:34.592470 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:34.592754 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":350,"threads_started":1}
W20260809 22:50:34.594544 21901 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:38989 (0 consecutive failures): Network error: Failed to ping master at 127.24.136.62:38989: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
I20260809 22:50:34.595392 21908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:34.595063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:34727)
0809 22:50:34.595199 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:34.595282 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 22:50:34.596266 21908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:34.595932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:37343)
0809 22:50:34.596066 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:34.596140 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 22:50:34.596508 21901 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:50:34.597209 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 21614
I20260809 22:50:34.597860 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:46755
--local_ip_for_outbound_sockets=127.24.136.2
--tserver_master_addrs=127.24.136.62:38989
--webserver_port=34511
--webserver_interface=127.24.136.2
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:50:34.960500 21913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:34.960809 21913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:34.960906 21913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:34.968508 21913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:34.968639 21913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:34.968775 21913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:50:34.979204 21913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:46755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=34511
--tserver_master_addrs=127.24.136.62:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:34.980574 21913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:34.981894 21913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:34.995574 21925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:34.995534 21927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:34.995548 21929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:34.996397 21913 server_base.cc:1034] running on GCE node
I20260809 22:50:34.997089 21913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:34.998178 21913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:34.999548 21913 hybrid_clock.cc:648] HybridClock initialized: now 1786315834999461 us; error 63 us; skew 500 ppm
I20260809 22:50:34.999917 21913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:34 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315834, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:35.003304 21913 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:50:35.005167 21913 webserver.cc:460] Webserver started at http://127.24.136.2:34511/ using document root <none> and password file <none>
I20260809 22:50:35.006017 21913 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:35.006153 21913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:35.012424 21913 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:50:35.015724 21945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:35.017359 21913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:50:35.017565 21913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "3a5ee0af3c8d4da984658c7dc26a657c"
format_stamp: "Formatted at 2026-08-09 22:49:48 on dist-test-slave-pww0"
server_key: "bda3acbe6e6ada2c9cefa5863d989436"
server_key_iv: "7bb0e993b1b7899d8876b4484ed39e60"
server_key_version: "encryptionkey@0"
I20260809 22:50:35.018659 21913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:35.055959 21913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:35.060587 21913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:35.060876 21913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:35.061968 21913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:35.063446 21913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:35.063597 21913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:50:35.063715 21913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:35.063794 21913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:35.096840 21913 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:46755
I20260809 22:50:35.096935 22134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:46755 every 8 connection(s)
I20260809 22:50:35.098368 21913 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:50:35.101807 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 21913
I20260809 22:50:35.102651 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:43681
--local_ip_for_outbound_sockets=127.24.136.3
--tserver_master_addrs=127.24.136.62:38989
--webserver_port=40319
--webserver_interface=127.24.136.3
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:50:35.103987 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.101746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:41971)
0809 22:50:35.102638 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:35.102979 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":347,"threads_started":1}
W20260809 22:50:35.104810 22137 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:38989 (0 consecutive failures): Network error: Failed to ping master at 127.24.136.62:38989: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
I20260809 22:50:35.105774 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.105398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:52723)
0809 22:50:35.105562 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:35.105634 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 22:50:35.106681 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.106351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:54067)
0809 22:50:35.106481 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:35.106561 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:50:35.106968 22137 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:50:35.458441 22142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:35.458791 22142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:35.458945 22142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:35.466776 22142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:35.466923 22142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:35.467088 22142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:50:35.477358 22142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:43681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=40319
--tserver_master_addrs=127.24.136.62:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:35.478811 22142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:35.480257 22142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:35.494171 22153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:35.494467 22157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:35.494753 22151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:35.495744 22142 server_base.cc:1034] running on GCE node
I20260809 22:50:35.496400 22142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:35.497468 22142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:35.498688 22142 hybrid_clock.cc:648] HybridClock initialized: now 1786315835498620 us; error 60 us; skew 500 ppm
I20260809 22:50:35.499032 22142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:35 dist-test-slave-pww0 krb5kdc[19613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315835, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:35.502650 22142 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:50:35.504451 22142 webserver.cc:460] Webserver started at http://127.24.136.3:40319/ using document root <none> and password file <none>
I20260809 22:50:35.505180 22142 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:35.505322 22142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:35.511420 22142 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:50:35.514676 22172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:35.516345 22142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:50:35.516526 22142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "78eb401e40f34f7eb81022428aef56d4"
format_stamp: "Formatted at 2026-08-09 22:49:49 on dist-test-slave-pww0"
server_key: "d199bb72824a7ab30b6b6e952d2df832"
server_key_iv: "937679f8720373dbd2302238fd6bb6d5"
server_key_version: "encryptionkey@0"
I20260809 22:50:35.517552 22142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:35.534739 22142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:35.539220 22142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:35.539551 22142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:35.540647 22142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:35.542197 22142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:35.542294 22142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:35.542399 22142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:35.542474 22142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:35.575333 22142 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:43681
I20260809 22:50:35.575409 22353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:43681 every 8 connection(s)
I20260809 22:50:35.576905 22142 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:50:35.582185 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.580123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:37603)
0809 22:50:35.580965 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:35.581226 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":331,"threads_started":1}
W20260809 22:50:35.583031 22354 heartbeater.cc:644] Failed to heartbeat to 127.24.136.62:38989 (0 consecutive failures): Network error: Failed to ping master at 127.24.136.62:38989: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
I20260809 22:50:35.583992 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.583652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:45047)
0809 22:50:35.583800 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:35.583875 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:50:35.584874 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.584490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:35399)
0809 22:50:35.584661 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:35.584748 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 22:50:35.585141 22354 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:50:35.585459 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 22142
I20260809 22:50:35.597975 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.597065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:54159)
0809 22:50:35.597637 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:35.597756 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":265,"threads_started":1}
I20260809 22:50:35.990680 22362 raft_consensus.cc:482] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:50:35.991010 22362 raft_consensus.cc:504] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:35.992810 22362 leader_election.cc:288] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681), 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
Aug 09 22:50:36 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315834, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
Aug 09 22:50:36 dist-test-slave-pww0 krb5kdc[19613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315834, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:50:36.020732 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.993432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:46755 (local address 127.24.136.1:46871)
0809 22:50:35.995647 (+ 2215us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:35.995790 (+ 143us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:35.995939 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:35.998952 (+ 3013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:35.998959 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:35.999734 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:36.001006 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:36.001044 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:36.004229 (+ 3185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:36.004236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:36.005496 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:36.005510 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:36.006861 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:36.006869 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:36.007048 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:36.007733 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:50:36.007752 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:36.010327 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:36.015393 (+ 5066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:36.015408 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:36.015429 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:36.016358 (+ 929us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:36.017028 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:36.017035 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:36.017038 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:36.017361 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:36.018606 (+ 1245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:36.018616 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:36.019173 (+ 557us) client_negotiation.cc:769] Sending connection context
0809 22:50:36.019557 (+ 384us) client_negotiation.cc:240] Negotiation successful
0809 22:50:36.020163 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1876,"mutex_wait_us":233,"spinlock_wait_cycles":80768,"thread_start_us":1618,"threads_started":1}
I20260809 22:50:36.020879 22367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.993873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:46871 (local address 127.24.136.2:46755)
0809 22:50:35.996503 (+ 2630us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:35.996514 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:35.996552 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:50:35.998094 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:35.998104 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:35.998562 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:35.999582 (+ 1020us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:36.002507 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:36.004021 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:36.005683 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:36.006637 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:36.006816 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:36.010726 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:36.010739 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:36.010757 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:36.010820 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:36.015127 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:36.016597 (+ 1470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:36.016609 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:36.016619 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:36.016891 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:36.017543 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:36.017550 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:36.017553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:36.018407 (+ 854us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:36.018652 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:36.019896 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0809 22:50:36.020382 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2455,"thread_start_us":2331,"threads_started":1}
I20260809 22:50:36.020879 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.993645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:49723 (local address 127.24.136.3:43681)
0809 22:50:35.995936 (+ 2291us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:35.995949 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:35.995989 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:50:35.997570 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:35.997580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:35.998027 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:35.998351 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:36.001317 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:36.002907 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:36.005012 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:36.005880 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:36.006097 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:36.010403 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:36.010420 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:36.010445 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:36.010509 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:36.015127 (+ 4618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:36.016603 (+ 1476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:36.016612 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:36.016619 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:36.016891 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:36.017561 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:36.017567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:36.017569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:36.018406 (+ 837us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:36.018652 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:36.019898 (+ 1246us) server_negotiation.cc:299] Negotiation successful
0809 22:50:36.020382 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2129,"thread_start_us":1849,"threads_started":1}
I20260809 22:50:36.020695 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:35.993323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:43681 (local address 127.24.136.1:49723)
0809 22:50:35.993746 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:35.993791 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:35.994470 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:35.998645 (+ 4175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:35.998658 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:35.999733 (+ 1075us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:36.001006 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:36.001044 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:36.003164 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:36.003170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:36.004816 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:36.004834 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:36.006128 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:36.006134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:36.006364 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:36.007146 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:50:36.007173 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:36.009935 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:36.015393 (+ 5458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:36.015408 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:36.015429 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:36.016358 (+ 929us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:36.017028 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:36.017034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:36.017038 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:36.017361 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:36.018606 (+ 1245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:36.018616 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:36.019173 (+ 557us) client_negotiation.cc:769] Sending connection context
0809 22:50:36.019557 (+ 384us) client_negotiation.cc:240] Negotiation successful
0809 22:50:36.020305 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":350,"spinlock_wait_cycles":76800}
W20260809 22:50:36.024737 21718 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
W20260809 22:50:36.025118 21716 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
I20260809 22:50:36.025271 21716 leader_election.cc:302] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13d1aa2ed32e4f07ba96dbd8113b8952; no voters: 3a5ee0af3c8d4da984658c7dc26a657c, 78eb401e40f34f7eb81022428aef56d4
I20260809 22:50:36.025739 22362 raft_consensus.cc:2726] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:50:36.108664 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:36.107713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:42775)
0809 22:50:36.108225 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:36.108385 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":231,"threads_started":1}
I20260809 22:50:36.586603 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:36.585778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:38479)
0809 22:50:36.586239 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:36.586367 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":215,"threads_started":1}
I20260809 22:50:36.599848 22371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:36.599048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:58273)
0809 22:50:36.599582 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:36.599676 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":274,"threads_started":1}
I20260809 22:50:37.110603 22378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:37.109798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:36555)
0809 22:50:37.110332 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:37.110430 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":259,"threads_started":1}
I20260809 22:50:37.588640 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:37.587724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:53395)
0809 22:50:37.588270 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:37.588404 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":271,"threads_started":1}
I20260809 22:50:37.601743 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:37.600836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:34973)
0809 22:50:37.601442 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:37.601541 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":293,"threads_started":1}
I20260809 22:50:38.046732 22381 raft_consensus.cc:482] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:50:38.046957 22381 raft_consensus.cc:504] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:38.047711 22381 leader_election.cc:288] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681), 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
W20260809 22:50:38.048825 21718 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
W20260809 22:50:38.049014 21716 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
I20260809 22:50:38.049163 21716 leader_election.cc:302] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13d1aa2ed32e4f07ba96dbd8113b8952; no voters: 3a5ee0af3c8d4da984658c7dc26a657c, 78eb401e40f34f7eb81022428aef56d4
I20260809 22:50:38.049432 22381 raft_consensus.cc:2726] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:50:38.112480 22382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:38.111707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:38349)
0809 22:50:38.112208 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:38.112304 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":252,"threads_started":1}
I20260809 22:50:38.590584 22383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:38.589709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:49645)
0809 22:50:38.590237 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:38.590376 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":266,"threads_started":1}
I20260809 22:50:38.603505 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:38.602719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:37509)
0809 22:50:38.603195 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:38.603293 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":237,"threads_started":1}
I20260809 22:50:39.114390 22386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:39.113552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:57217)
0809 22:50:39.114098 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:39.114197 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":278,"threads_started":1}
I20260809 22:50:39.592402 22387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:39.591641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:58919)
0809 22:50:39.592107 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:39.592201 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":221,"threads_started":1}
I20260809 22:50:39.605417 22388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:39.604544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:46539)
0809 22:50:39.605121 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:39.605227 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":277,"threads_started":1}
I20260809 22:50:40.116127 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:40.115358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:35911)
0809 22:50:40.115866 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:40.115961 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":263,"threads_started":1}
I20260809 22:50:40.594394 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:40.593449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:38615)
0809 22:50:40.594131 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:40.594224 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":329,"threads_started":1}
I20260809 22:50:40.607260 22393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:40.606405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:43351)
0809 22:50:40.606961 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:40.607065 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":332,"threads_started":1}
I20260809 22:50:41.091931 22394 raft_consensus.cc:482] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:50:41.092156 22394 raft_consensus.cc:504] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:41.092875 22394 leader_election.cc:288] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681), 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
W20260809 22:50:41.093981 21718 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
W20260809 22:50:41.094179 21716 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
I20260809 22:50:41.094305 21716 leader_election.cc:302] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13d1aa2ed32e4f07ba96dbd8113b8952; no voters: 3a5ee0af3c8d4da984658c7dc26a657c, 78eb401e40f34f7eb81022428aef56d4
I20260809 22:50:41.094570 22394 raft_consensus.cc:2726] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:50:41.117991 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:41.117158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:36091)
0809 22:50:41.117711 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:41.117806 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":316,"threads_started":1}
I20260809 22:50:41.596108 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:41.595267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:41899)
0809 22:50:41.595818 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:41.595915 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":322,"threads_started":1}
I20260809 22:50:41.609010 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:41.608231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:59817)
0809 22:50:41.608715 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:41.608808 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":255,"threads_started":1}
I20260809 22:50:42.119923 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:42.118970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:39495)
0809 22:50:42.119609 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:42.119721 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":279,"threads_started":1}
I20260809 22:50:42.597906 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:42.597052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:32841)
0809 22:50:42.597599 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:42.597690 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":248,"threads_started":1}
I20260809 22:50:42.610790 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:42.610003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:51839)
0809 22:50:42.610531 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:42.610624 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":267,"threads_started":1}
I20260809 22:50:42.784104 22403 raft_consensus.cc:482] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:50:42.784369 22403 raft_consensus.cc:504] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:42.785133 22403 leader_election.cc:288] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681), 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
W20260809 22:50:42.786226 21716 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
W20260809 22:50:42.786407 21718 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
I20260809 22:50:42.786620 21718 leader_election.cc:302] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13d1aa2ed32e4f07ba96dbd8113b8952; no voters: 3a5ee0af3c8d4da984658c7dc26a657c, 78eb401e40f34f7eb81022428aef56d4
I20260809 22:50:42.786888 22403 raft_consensus.cc:2726] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:50:43.121699 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:43.120966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:44907)
0809 22:50:43.121409 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:43.121507 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":221,"threads_started":1}
I20260809 22:50:43.599839 22406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:43.598940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:46367)
0809 22:50:43.599527 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:43.599636 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":277,"threads_started":1}
I20260809 22:50:43.612666 22407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:43.611844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:48955)
0809 22:50:43.612358 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:43.612468 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":264,"threads_started":1}
I20260809 22:50:44.123728 22408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:44.122789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:43733)
0809 22:50:44.123393 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:44.123547 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":277,"threads_started":1}
I20260809 22:50:44.601783 22409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:44.600916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:44069)
0809 22:50:44.601486 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:44.601589 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":256,"threads_started":1}
I20260809 22:50:44.614836 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:44.613898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:50327)
0809 22:50:44.614475 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:44.614605 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":354,"threads_started":1}
I20260809 22:50:45.026503 22412 raft_consensus.cc:482] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:50:45.026722 22412 raft_consensus.cc:504] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78eb401e40f34f7eb81022428aef56d4" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 43681 } } peers { permanent_uuid: "13d1aa2ed32e4f07ba96dbd8113b8952" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 44493 } } peers { permanent_uuid: "3a5ee0af3c8d4da984658c7dc26a657c" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 46755 } }
I20260809 22:50:45.027428 22412 leader_election.cc:288] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681), 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755)
W20260809 22:50:45.028469 21718 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a5ee0af3c8d4da984658c7dc26a657c (127.24.136.2:46755): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
W20260809 22:50:45.028650 21716 leader_election.cc:341] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78eb401e40f34f7eb81022428aef56d4 (127.24.136.3:43681): Not found: Tablet not found: ff6f1bb5eb2043c19a4375b7d803831f
I20260809 22:50:45.028774 21716 leader_election.cc:302] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13d1aa2ed32e4f07ba96dbd8113b8952; no voters: 3a5ee0af3c8d4da984658c7dc26a657c, 78eb401e40f34f7eb81022428aef56d4
I20260809 22:50:45.029008 22412 raft_consensus.cc:2726] T ff6f1bb5eb2043c19a4375b7d803831f P 13d1aa2ed32e4f07ba96dbd8113b8952 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:50:45.125928 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:45.124798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:43833)
0809 22:50:45.125512 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:45.125641 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":415,"threads_started":1}
I20260809 22:50:45.603646 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:45.602837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:39113)
0809 22:50:45.603315 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:45.603415 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260809 22:50:45.616778 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:45.615948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:36915)
0809 22:50:45.616469 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:45.616579 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":291,"threads_started":1}
I20260809 22:50:46.127907 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:46.127015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:55691)
0809 22:50:46.127613 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:46.127733 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":274,"threads_started":1}
I20260809 22:50:46.605517 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:46.604712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:36375)
0809 22:50:46.605245 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:46.605350 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":251,"threads_started":1}
I20260809 22:50:46.618621 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:46.617788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:39233)
0809 22:50:46.618327 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:46.618426 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":289,"threads_started":1}
I20260809 22:50:47.129875 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:47.128954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:45519)
0809 22:50:47.129537 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:47.129663 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":298,"threads_started":1}
I20260809 22:50:47.607514 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:47.606661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:48251)
0809 22:50:47.607160 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:47.607285 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":249,"threads_started":1}
I20260809 22:50:47.620563 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:47.619683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:36379)
0809 22:50:47.620235 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:47.620365 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":296,"threads_started":1}
I20260809 22:50:48.132021 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:48.131156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:49771)
0809 22:50:48.131720 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:48.131833 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":303,"threads_started":1}
I20260809 22:50:48.609362 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:48.608566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:40519)
0809 22:50:48.609068 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:48.609169 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":247,"threads_started":1}
I20260809 22:50:48.622596 22433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:48.621644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:44905)
0809 22:50:48.622171 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:48.622292 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":266,"threads_started":1}
I20260809 22:50:49.140379 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:49.133122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:59835)
0809 22:50:49.140011 (+ 6889us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:49.140141 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6748,"thread_start_us":6604,"threads_started":1}
I20260809 22:50:49.611312 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:49.610418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:59241)
0809 22:50:49.611018 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:49.611105 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":281,"threads_started":1}
I20260809 22:50:49.624747 22439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:49.623944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:51579)
0809 22:50:49.624457 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:49.624564 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":253,"threads_started":1}
I20260809 22:50:50.142534 22441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:50.141746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.2:41447)
0809 22:50:50.142250 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:50.142357 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":253,"threads_started":1}
I20260809 22:50:50.592093 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 21614
I20260809 22:50:50.609236 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:50.593338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48282 (local address 127.24.136.1:44493)
0809 22:50:50.594097 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:50.594108 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:50.594146 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:50:50.594230 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:50.594239 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:50.594524 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:50.594695 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:50.596641 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:50.597735 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:50.599186 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:50.599881 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:50.600003 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:50.602794 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:50.602809 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:50.602826 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:50.602869 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:50.605986 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:50.606865 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:50.606874 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:50.606877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:50.607058 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:50.607509 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:50.607515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:50.607518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:50.607958 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:50.608226 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:50.608606 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 22:50:50.608878 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":412,"threads_started":1}
I20260809 22:50:50.610675 21895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:50.614854 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:50.612651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.3:38573)
0809 22:50:50.614552 (+ 1901us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:50.614689 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1767,"thread_start_us":1643,"threads_started":1}
I20260809 22:50:50.701936 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 21614
I20260809 22:50:50.702072 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:50.700957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:38989 (local address 127.24.136.1:49263)
0809 22:50:50.701552 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:50.701772 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.62:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":289,"threads_started":1}
I20260809 22:50:50.738405 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 21913
I20260809 22:50:50.753305 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:50.739221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58400 (local address 127.24.136.2:46755)
0809 22:50:50.739715 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:50.739720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:50.739738 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:50:50.739797 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:50.739801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:50.739865 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:50.739963 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:50.741343 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:50.742092 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:50.743563 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:50.744010 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:50.744114 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:50.746857 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:50.746862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:50.746866 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:50.746907 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:50.749761 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:50.750826 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:50.750832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:50.750834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:50.750992 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:50.751398 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:50.751403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:50.751405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:50.751878 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:50.752128 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:50.752485 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 22:50:50.752708 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":271,"threads_started":1}
I20260809 22:50:50.754184 22124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:50.840893 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 21913
I20260809 22:50:50.864916 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 22142
I20260809 22:50:50.879132 22453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:50.865921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52864 (local address 127.24.136.3:43681)
0809 22:50:50.866541 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:50.866546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:50.866584 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:50:50.866644 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:50.866648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:50.866721 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:50.866816 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:50.868219 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:50.869081 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:50.870343 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:50.870861 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:50.870986 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:50.873767 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:50.873773 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:50.873776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:50.873815 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:50.876685 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:50.877385 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:50.877390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:50.877392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:50.877526 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:50.877892 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:50.877895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:50.877897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:50.878204 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:50.878419 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:50.878628 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 22:50:50.878815 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":329,"threads_started":1}
I20260809 22:50:50.879931 22346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:50.965893 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 22142
2026-08-09T22:50:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65335 ms)
[----------] 30 tests from SecurityITest (317435 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:50:51 dist-test-slave-pww0 krb5kdc[22457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:50:51 dist-test-slave-pww0 krb5kdc[22457](info): set up 2 sockets
Aug 09 22:50:51 dist-test-slave-pww0 krb5kdc[22457](info): commencing operation
krb5kdc: starting...
W20260809 22:50:52.261981 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:50:52 dist-test-slave-pww0 krb5kdc[22457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:50:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:50:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:50:52.590925 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:42387
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:42387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:50:52.961894 22481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:52.962218 22481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:52.962319 22481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:52.970191 22481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:50:52.970357 22481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:52.970435 22481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:52.970515 22481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:50:52.970563 22481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:50:52.981143 22481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:42387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:42387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:52.982875 22481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:52.984498 22481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:52.997743 22492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:52.997700 22495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:52.997767 22493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:52.998558 22481 server_base.cc:1034] running on GCE node
I20260809 22:50:52.999501 22481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:53.001237 22481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:53.002482 22481 hybrid_clock.cc:648] HybridClock initialized: now 1786315853002417 us; error 57 us; skew 500 ppm
I20260809 22:50:53.002880 22481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:52 dist-test-slave-pww0 krb5kdc[22457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315852, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:53.006589 22481 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:50:53.008574 22481 webserver.cc:460] Webserver started at http://127.24.136.62:45963/ using document root <none> and password file <none>
I20260809 22:50:53.009359 22481 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:53.009480 22481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:53.009819 22481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:50:53.013260 22481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "33a93581187d47e0b9d41665cff134f2"
format_stamp: "Formatted at 2026-08-09 22:50:53 on dist-test-slave-pww0"
server_key: "f2109b6d46c2c5be9725392fc7fb2519"
server_key_iv: "6d5a0c145b212de632c3e4a5b6080ef3"
server_key_version: "encryptionkey@0"
I20260809 22:50:53.014118 22481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "33a93581187d47e0b9d41665cff134f2"
format_stamp: "Formatted at 2026-08-09 22:50:53 on dist-test-slave-pww0"
server_key: "f2109b6d46c2c5be9725392fc7fb2519"
server_key_iv: "6d5a0c145b212de632c3e4a5b6080ef3"
server_key_version: "encryptionkey@0"
I20260809 22:50:53.020046 22481 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 22:50:53.024042 22507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:53.025545 22481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:50:53.025745 22481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "33a93581187d47e0b9d41665cff134f2"
format_stamp: "Formatted at 2026-08-09 22:50:53 on dist-test-slave-pww0"
server_key: "f2109b6d46c2c5be9725392fc7fb2519"
server_key_iv: "6d5a0c145b212de632c3e4a5b6080ef3"
server_key_version: "encryptionkey@0"
I20260809 22:50:53.025970 22481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:53.039964 22481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:53.044732 22481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:53.045044 22481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:53.064785 22481 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:42387
I20260809 22:50:53.064805 22604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:42387 every 8 connection(s)
I20260809 22:50:53.066341 22481 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:50:53.070538 22605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:53.075032 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 22481
I20260809 22:50:53.075268 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:50:53.075992 25120 external_mini_cluster.cc:1428] Setting key d83ab1476ce8ef94bd0f1305edd10f33
I20260809 22:50:53.082770 22605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2: Bootstrap starting.
I20260809 22:50:53.086804 22605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:53.088148 22605 log.cc:824] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:53.091988 22605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2: No bootstrap required, opened a new log
Aug 09 22:50:53 dist-test-slave-pww0 krb5kdc[22457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:50:53.102820 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:53.078224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45810 (local address 127.24.136.62:42387)
0809 22:50:53.079061 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:53.079074 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:53.079132 (+ 58us) server_negotiation.cc:407] Connection header received
0809 22:50:53.081106 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:53.081135 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:53.081579 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:53.081922 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:53.084036 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:53.085449 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:53.087482 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:53.088244 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:53.088457 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:53.093449 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:53.093462 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:53.093485 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:53.093543 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:53.097733 (+ 4190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:53.098564 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:53.098574 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:53.098584 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:53.098814 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:53.099193 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:53.099199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:53.099202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:53.100028 (+ 826us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:53.100376 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:53.100818 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 22:50:53.101359 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":365,"threads_started":1}
I20260809 22:50:53.102849 22605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a93581187d47e0b9d41665cff134f2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42387 } }
I20260809 22:50:53.103370 22605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:53.103521 22605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a93581187d47e0b9d41665cff134f2, State: Initialized, Role: FOLLOWER
I20260809 22:50:53.104142 22605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a93581187d47e0b9d41665cff134f2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42387 } }
I20260809 22:50:53.104387 22605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:50:53.104540 22605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:50:53.104744 22605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:53.107606 22605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a93581187d47e0b9d41665cff134f2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42387 } }
I20260809 22:50:53.108139 22605 leader_election.cc:302] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33a93581187d47e0b9d41665cff134f2; no voters:
I20260809 22:50:53.108718 22605 leader_election.cc:288] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:50:53.108878 22612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:50:53.109529 22612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [term 1 LEADER]: Becoming Leader. State: Replica: 33a93581187d47e0b9d41665cff134f2, State: Running, Role: LEADER
I20260809 22:50:53.110073 22612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a93581187d47e0b9d41665cff134f2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42387 } }
I20260809 22:50:53.110776 22605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:50:53.113293 22613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33a93581187d47e0b9d41665cff134f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a93581187d47e0b9d41665cff134f2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42387 } } }
I20260809 22:50:53.113791 22613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:50:53.113327 22614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33a93581187d47e0b9d41665cff134f2. Latest consensus state: current_term: 1 leader_uuid: "33a93581187d47e0b9d41665cff134f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33a93581187d47e0b9d41665cff134f2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42387 } } }
I20260809 22:50:53.114619 22614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:50:53.115660 22619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:50:53.121584 22619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:50:53.138219 22633 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:50:53.139541 22619 catalog_manager.cc:1347] Generated new cluster ID: 059d4d334c014fddbb49bdb1f67b6b81
I20260809 22:50:53.139729 22619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:50:53.158759 22619 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:50:53.160763 22619 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:50:53.173007 22619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2: Generated new TSK 0
I20260809 22:50:53.174000 22619 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:50:53.267669 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42387
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:50:53.678292 22640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:53.678647 22640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:53.678766 22640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:53.686347 22640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:53.686470 22640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:53.686584 22640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:50:53.697069 22640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:53.698498 22640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:53.699858 22640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:53.713240 22659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:53.713281 22653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:53.713318 22655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:53.714165 22640 server_base.cc:1034] running on GCE node
I20260809 22:50:53.714869 22640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:53.715977 22640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:53.717188 22640 hybrid_clock.cc:648] HybridClock initialized: now 1786315853717143 us; error 45 us; skew 500 ppm
I20260809 22:50:53.717564 22640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:53 dist-test-slave-pww0 krb5kdc[22457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315853, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:53.720866 22640 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:50:53.722462 22640 webserver.cc:460] Webserver started at http://127.24.136.1:43987/ using document root <none> and password file <none>
I20260809 22:50:53.723173 22640 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:53.723285 22640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:53.723639 22640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:50:53.726769 22640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "f6a89b0c768042da89619795a57bb0fa"
format_stamp: "Formatted at 2026-08-09 22:50:53 on dist-test-slave-pww0"
server_key: "53786805615614965d3e53eecaa0a4cd"
server_key_iv: "d9b1776aa21bd110b135320ad6429845"
server_key_version: "encryptionkey@0"
I20260809 22:50:53.727669 22640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "f6a89b0c768042da89619795a57bb0fa"
format_stamp: "Formatted at 2026-08-09 22:50:53 on dist-test-slave-pww0"
server_key: "53786805615614965d3e53eecaa0a4cd"
server_key_iv: "d9b1776aa21bd110b135320ad6429845"
server_key_version: "encryptionkey@0"
I20260809 22:50:53.733599 22640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:50:53.737754 22671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:53.739281 22640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:50:53.739545 22640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "f6a89b0c768042da89619795a57bb0fa"
format_stamp: "Formatted at 2026-08-09 22:50:53 on dist-test-slave-pww0"
server_key: "53786805615614965d3e53eecaa0a4cd"
server_key_iv: "d9b1776aa21bd110b135320ad6429845"
server_key_version: "encryptionkey@0"
I20260809 22:50:53.739795 22640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:53.754598 22640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:53.759204 22640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:53.759539 22640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:53.760623 22640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:53.762251 22640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:53.762346 22640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:53.762470 22640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:53.762531 22640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:53.797174 22640 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:39705
I20260809 22:50:53.797224 22866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:39705 every 8 connection(s)
I20260809 22:50:53.798509 22640 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:50:53.798903 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 22640
I20260809 22:50:53.799113 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:50:53.799938 25120 external_mini_cluster.cc:1428] Setting key 7952422f4b7c3ebc771479c4e08a8ee7
Aug 09 22:50:53 dist-test-slave-pww0 krb5kdc[22457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315853, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:50:53.822170 22870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:53.801405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:57775 (local address 127.24.136.62:42387)
0809 22:50:53.801906 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:53.801912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:53.803047 (+ 1135us) server_negotiation.cc:407] Connection header received
0809 22:50:53.805565 (+ 2518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:53.805570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:53.805644 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:53.805749 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:53.807900 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:53.808645 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:53.810996 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:53.811566 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:53.811695 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:53.815981 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:53.815987 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:53.815991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:53.816031 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:53.818673 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:53.819581 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:53.819586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:53.819588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:53.819736 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:53.820233 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:53.820238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:53.820240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:53.820547 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:53.820690 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:53.821551 (+ 861us) server_negotiation.cc:299] Negotiation successful
0809 22:50:53.821813 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":252,"threads_started":1}
I20260809 22:50:53.823223 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:53.801994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42387 (local address 127.24.136.1:57775)
0809 22:50:53.802841 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:53.802907 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:53.805199 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:53.806045 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:53.806058 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:53.806833 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:53.807702 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:53.807730 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:53.808818 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:53.808823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:53.810784 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:53.810802 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:53.811839 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:53.811846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:53.812149 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:53.812932 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:50:53.812963 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:53.815685 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:53.818877 (+ 3192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:53.818886 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:53.818903 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:53.819401 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:53.819870 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:53.819875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:53.819877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:53.820096 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:53.820694 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:53.820704 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:53.821129 (+ 425us) client_negotiation.cc:769] Sending connection context
0809 22:50:53.821393 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 22:50:53.822006 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":298,"threads_started":1}
I20260809 22:50:53.825122 22867 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42387
I20260809 22:50:53.825489 22867 heartbeater.cc:464] Registering TS with master...
I20260809 22:50:53.826346 22867 heartbeater.cc:511] Master 127.24.136.62:42387 requested a full tablet report, sending...
I20260809 22:50:53.829070 22537 ts_manager.cc:194] Registered new tserver with Master: f6a89b0c768042da89619795a57bb0fa (127.24.136.1:39705)
I20260809 22:50:53.831530 22537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:57775
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:50:53.872848 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42387
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:50:54.241197 22877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:54.241501 22877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:54.241624 22877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:54.249480 22877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:54.249612 22877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:54.249770 22877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:50:54.260234 22877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:54.261656 22877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:54.263099 22877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:54.277431 22887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:54.277474 22886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:54.277765 22877 server_base.cc:1034] running on GCE node
W20260809 22:50:54.277499 22890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:54.278823 22877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:54.279917 22877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:54.281147 22877 hybrid_clock.cc:648] HybridClock initialized: now 1786315854281082 us; error 50 us; skew 500 ppm
I20260809 22:50:54.281530 22877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:54 dist-test-slave-pww0 krb5kdc[22457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315854, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:54.288921 22877 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:50:54.290740 22877 webserver.cc:460] Webserver started at http://127.24.136.2:42755/ using document root <none> and password file <none>
I20260809 22:50:54.291505 22877 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:54.291653 22877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:54.291980 22877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:50:54.295181 22877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "5870f0fca3f3461d896040b6b582c2c4"
format_stamp: "Formatted at 2026-08-09 22:50:54 on dist-test-slave-pww0"
server_key: "1befa59039827961b46fe04077477ae0"
server_key_iv: "8315f56c50731ce53bb73f865ad69300"
server_key_version: "encryptionkey@0"
I20260809 22:50:54.296080 22877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "5870f0fca3f3461d896040b6b582c2c4"
format_stamp: "Formatted at 2026-08-09 22:50:54 on dist-test-slave-pww0"
server_key: "1befa59039827961b46fe04077477ae0"
server_key_iv: "8315f56c50731ce53bb73f865ad69300"
server_key_version: "encryptionkey@0"
I20260809 22:50:54.301887 22877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:50:54.305876 22903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:54.307637 22877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:50:54.307852 22877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "5870f0fca3f3461d896040b6b582c2c4"
format_stamp: "Formatted at 2026-08-09 22:50:54 on dist-test-slave-pww0"
server_key: "1befa59039827961b46fe04077477ae0"
server_key_iv: "8315f56c50731ce53bb73f865ad69300"
server_key_version: "encryptionkey@0"
I20260809 22:50:54.308096 22877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:54.330236 22877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:54.335008 22877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:54.335335 22877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:54.336455 22877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:54.338021 22877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:54.338111 22877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:54.338239 22877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:54.338301 22877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:54.371522 22877 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:43393
I20260809 22:50:54.371588 23096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:43393 every 8 connection(s)
I20260809 22:50:54.372885 22877 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:50:54.376713 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 22877
I20260809 22:50:54.376940 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:50:54.377527 25120 external_mini_cluster.cc:1428] Setting key 31c58fba13a8534b9e45ca6a5d6d50ca
Aug 09 22:50:54 dist-test-slave-pww0 krb5kdc[22457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315854, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:50:54.397656 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:54.375795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:59597 (local address 127.24.136.62:42387)
0809 22:50:54.376287 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:54.376293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:54.377588 (+ 1295us) server_negotiation.cc:407] Connection header received
0809 22:50:54.380091 (+ 2503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:54.380097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:54.380186 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:54.380315 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:54.382130 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.383143 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.384975 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.385565 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.385700 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:54.390113 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:54.390120 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:54.390124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:54.390174 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:54.393633 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:54.394893 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:54.394898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:54.394901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:54.395057 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:54.395674 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:54.395679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:54.395682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:54.396011 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:54.396149 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:54.397161 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0809 22:50:54.397377 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":235,"threads_started":1}
I20260809 22:50:54.399040 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:54.376475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42387 (local address 127.24.136.2:59597)
0809 22:50:54.377393 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:54.377444 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:54.379714 (+ 2270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:54.380611 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:54.380621 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:54.381229 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:54.381892 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:54.381915 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.383370 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.383374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:54.384792 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:54.384806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.385843 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.385848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:54.386093 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:54.386922 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:50:54.386957 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:54.389650 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:54.393889 (+ 4239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:54.393903 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:54.393928 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:54.394686 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:54.395197 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:54.395203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:54.395206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:54.395523 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:54.396169 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:54.396178 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:54.396679 (+ 501us) client_negotiation.cc:769] Sending connection context
0809 22:50:54.397006 (+ 327us) client_negotiation.cc:240] Negotiation successful
0809 22:50:54.397620 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":394,"threads_started":1}
I20260809 22:50:54.400797 23097 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42387
I20260809 22:50:54.401150 23097 heartbeater.cc:464] Registering TS with master...
I20260809 22:50:54.402073 23097 heartbeater.cc:511] Master 127.24.136.62:42387 requested a full tablet report, sending...
I20260809 22:50:54.403936 22537 ts_manager.cc:194] Registered new tserver with Master: 5870f0fca3f3461d896040b6b582c2c4 (127.24.136.2:43393)
I20260809 22:50:54.405035 22537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:59597
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:50:54.453009 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42387
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:50:54.821552 23107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:54.821887 23107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:54.821986 23107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:54.829679 23107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:54.829815 23107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:54.829960 23107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:50:54.835357 22867 heartbeater.cc:503] Master 127.24.136.62:42387 was elected leader, sending a full tablet report...
I20260809 22:50:54.840862 23107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:54.842300 23107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:54.843827 23107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:54.857889 23113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:54.857903 23116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:54.858651 23118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:54.858987 23107 server_base.cc:1034] running on GCE node
I20260809 22:50:54.859768 23107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:54.860862 23107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:54.862128 23107 hybrid_clock.cc:648] HybridClock initialized: now 1786315854862070 us; error 48 us; skew 500 ppm
I20260809 22:50:54.862493 23107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:50:54 dist-test-slave-pww0 krb5kdc[22457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315854, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:50:54.866359 23107 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:50:54.868297 23107 webserver.cc:460] Webserver started at http://127.24.136.3:34545/ using document root <none> and password file <none>
I20260809 22:50:54.869068 23107 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:54.869186 23107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:54.869482 23107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:50:54.872845 23107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "b797ffd76aa3443da6d166e7a38d992f"
format_stamp: "Formatted at 2026-08-09 22:50:54 on dist-test-slave-pww0"
server_key: "222edf303e82e036d2dc7ac62daf2743"
server_key_iv: "e6b77d9f73d4e3f007817037dd34c920"
server_key_version: "encryptionkey@0"
I20260809 22:50:54.873673 23107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "b797ffd76aa3443da6d166e7a38d992f"
format_stamp: "Formatted at 2026-08-09 22:50:54 on dist-test-slave-pww0"
server_key: "222edf303e82e036d2dc7ac62daf2743"
server_key_iv: "e6b77d9f73d4e3f007817037dd34c920"
server_key_version: "encryptionkey@0"
I20260809 22:50:54.879520 23107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:50:54.883661 23136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:54.885315 23107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:50:54.885504 23107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "b797ffd76aa3443da6d166e7a38d992f"
format_stamp: "Formatted at 2026-08-09 22:50:54 on dist-test-slave-pww0"
server_key: "222edf303e82e036d2dc7ac62daf2743"
server_key_iv: "e6b77d9f73d4e3f007817037dd34c920"
server_key_version: "encryptionkey@0"
I20260809 22:50:54.885713 23107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:54.899361 23107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:54.904233 23107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:54.904539 23107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:54.905581 23107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:54.907064 23107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:54.907148 23107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:54.907260 23107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:54.907313 23107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:54.940793 23107 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:33237
I20260809 22:50:54.940867 23335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:33237 every 8 connection(s)
I20260809 22:50:54.942147 23107 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:50:54.948102 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 23107
I20260809 22:50:54.948277 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:50:54.948864 25120 external_mini_cluster.cc:1428] Setting key 0804f51a14a8ca1cf8f650ec07850d69
Aug 09 22:50:54 dist-test-slave-pww0 krb5kdc[22457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315854, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:50:54.964145 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:54.944969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:52803 (local address 127.24.136.62:42387)
0809 22:50:54.945508 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:54.945513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:54.946565 (+ 1052us) server_negotiation.cc:407] Connection header received
0809 22:50:54.948595 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:54.948602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:54.948719 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:54.948861 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:54.950856 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.951939 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.953693 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.954118 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.954219 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:54.958072 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:54.958077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:54.958080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:54.958117 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:54.960770 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:54.961628 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:54.961632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:54.961634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:54.961764 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:54.962185 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:54.962189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:54.962192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:54.962501 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:54.962652 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:54.963447 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 22:50:54.963769 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":276,"threads_started":1}
I20260809 22:50:54.965190 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:54.945581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42387 (local address 127.24.136.3:52803)
0809 22:50:54.946361 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:54.946411 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:54.948228 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:54.949146 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:54.949156 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:54.949857 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:54.950643 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:54.950665 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.952126 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.952132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:54.953530 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:54.953560 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.954359 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.954364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:54.954615 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:54.955336 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:50:54.955361 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:54.957815 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:54.960955 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:54.960965 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:54.960981 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:54.961515 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:54.961849 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:50:54.961852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:50:54.961854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:50:54.962046 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:50:54.962662 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:54.962671 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:54.963030 (+ 359us) client_negotiation.cc:769] Sending connection context
0809 22:50:54.963322 (+ 292us) client_negotiation.cc:240] Negotiation successful
0809 22:50:54.963898 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":287,"threads_started":1}
I20260809 22:50:54.966596 23337 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42387
I20260809 22:50:54.966926 23337 heartbeater.cc:464] Registering TS with master...
I20260809 22:50:54.967730 23337 heartbeater.cc:511] Master 127.24.136.62:42387 requested a full tablet report, sending...
I20260809 22:50:54.969326 22537 ts_manager.cc:194] Registered new tserver with Master: b797ffd76aa3443da6d166e7a38d992f (127.24.136.3:33237)
I20260809 22:50:54.970153 22537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:52803
I20260809 22:50:54.978286 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:50:54.997767 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:54.983392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.24.136.62:42387)
0809 22:50:54.983641 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:54.983648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:54.984104 (+ 456us) server_negotiation.cc:407] Connection header received
0809 22:50:54.984299 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:54.984305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:54.984385 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:54.984477 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:54.986050 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.986826 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.988403 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:54.988825 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:54.988951 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:54.991839 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:54.991845 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:54.991848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:54.991891 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:54.994591 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:54.995486 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:54.995493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:54.995497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:54.995654 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:54.996088 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:54.996092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:54.996094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:54.996487 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:54.996727 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:54.997184 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 22:50:54.997450 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 22:50:55.002344 22537 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:50:55.009585 22537 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:50:55.050527 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.023893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:43393 (local address 127.0.0.1:43236)
0809 22:50:55.040239 (+ 16346us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:55.040302 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:55.040546 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:55.042080 (+ 1534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:55.042090 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:55.042129 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:55.042680 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.042694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.044699 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.044705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.047305 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.047321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.049313 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.049319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.049557 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.049627 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 22:50:55.049779 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 22:50:55.050032 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16063,"mutex_wait_us":10,"thread_start_us":15897,"threads_started":1}
I20260809 22:50:55.050586 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.023931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43236 (local address 127.24.136.2:43393)
0809 22:50:55.039564 (+ 15633us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:55.039578 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:55.041056 (+ 1478us) server_negotiation.cc:407] Connection header received
0809 22:50:55.041184 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:55.041193 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:55.041498 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:55.041715 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:55.042900 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.044541 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.047540 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.049103 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.049269 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.049500 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:55.050044 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 22:50:55.050208 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15347,"thread_start_us":16821,"threads_started":1}
I20260809 22:50:55.050575 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.024943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47456 (local address 127.24.136.1:39705)
0809 22:50:55.040613 (+ 15670us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:55.040624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:55.041310 (+ 686us) server_negotiation.cc:407] Connection header received
0809 22:50:55.041420 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:55.041429 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:55.041678 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:55.041901 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:55.043927 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.045560 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.047518 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.049112 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.049259 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.049526 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:55.050006 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 22:50:55.050202 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15435,"thread_start_us":16102,"threads_started":1}
I20260809 22:50:55.051664 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.024301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.0.0.1:47456)
0809 22:50:55.041095 (+ 16794us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:55.041114 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:55.041272 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:55.042266 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:55.042271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:55.042320 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:55.043677 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.043702 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.045704 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.045708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.047286 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.047304 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.049654 (+ 2350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.049659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.049796 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.049820 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:50:55.049903 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 22:50:55.050905 (+ 1002us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16676,"mutex_wait_us":1712,"spinlock_wait_cycles":5504,"thread_start_us":16528,"threads_started":1}
I20260809 22:50:55.052102 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.042500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:33237 (local address 127.0.0.1:34684)
0809 22:50:55.043320 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:55.043346 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:55.043497 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:55.044751 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:55.044756 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:55.044789 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:55.045255 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.045267 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.047318 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.047323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.049192 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.049210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.051288 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.051295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.051497 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.051532 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:50:55.051639 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 22:50:55.051789 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":491,"threads_started":1}
I20260809 22:50:55.053493 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.041238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.24.136.3:33237)
0809 22:50:55.044150 (+ 2912us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:55.044158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:55.044190 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:50:55.044293 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:55.044303 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:55.044515 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:55.044886 (+ 371us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:55.045471 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.047129 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.049422 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.050949 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.051093 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.051344 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:55.052212 (+ 868us) server_negotiation.cc:299] Negotiation successful
0809 22:50:55.053233 (+ 1021us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2743,"thread_start_us":2578,"threads_started":1}
I20260809 22:50:55.055775 22757 tablet_service.cc:1467] Processing CreateTablet for tablet b5218aa3e11e469d8ccc32ed2a03a22e (DEFAULT_TABLE table=test-table [id=f54b43582b4d45318aa926ad9b4193f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:50:55.055583 22979 tablet_service.cc:1467] Processing CreateTablet for tablet b5218aa3e11e469d8ccc32ed2a03a22e (DEFAULT_TABLE table=test-table [id=f54b43582b4d45318aa926ad9b4193f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:50:55.057236 22757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5218aa3e11e469d8ccc32ed2a03a22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:55.057236 22979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5218aa3e11e469d8ccc32ed2a03a22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:55.056748 23219 tablet_service.cc:1467] Processing CreateTablet for tablet b5218aa3e11e469d8ccc32ed2a03a22e (DEFAULT_TABLE table=test-table [id=f54b43582b4d45318aa926ad9b4193f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:50:55.058037 23219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5218aa3e11e469d8ccc32ed2a03a22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:55.078831 23377 tablet_bootstrap.cc:492] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: Bootstrap starting.
I20260809 22:50:55.078960 23376 tablet_bootstrap.cc:492] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: Bootstrap starting.
I20260809 22:50:55.083168 23377 tablet_bootstrap.cc:654] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:55.084278 23374 tablet_bootstrap.cc:492] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: Bootstrap starting.
I20260809 22:50:55.084607 23376 tablet_bootstrap.cc:654] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:55.086194 23376 log.cc:824] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:55.087913 23377 log.cc:824] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:55.088414 23374 tablet_bootstrap.cc:654] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:55.090013 23374 log.cc:824] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:55.093907 23374 tablet_bootstrap.cc:492] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: No bootstrap required, opened a new log
I20260809 22:50:55.093995 23377 tablet_bootstrap.cc:492] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: No bootstrap required, opened a new log
I20260809 22:50:55.094218 23374 ts_tablet_manager.cc:1397] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 22:50:55.094306 23377 ts_tablet_manager.cc:1397] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.000s
I20260809 22:50:55.094677 23376 tablet_bootstrap.cc:492] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: No bootstrap required, opened a new log
I20260809 22:50:55.094971 23376 ts_tablet_manager.cc:1397] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260809 22:50:55.102325 23374 raft_consensus.cc:348] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:50:55.102753 23374 raft_consensus.cc:374] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:55.102880 23374 raft_consensus.cc:729] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b797ffd76aa3443da6d166e7a38d992f, State: Initialized, Role: FOLLOWER
I20260809 22:50:55.103581 23374 consensus_queue.cc:260] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:50:55.104890 23337 heartbeater.cc:503] Master 127.24.136.62:42387 was elected leader, sending a full tablet report...
I20260809 22:50:55.105700 23374 ts_tablet_manager.cc:1428] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 22:50:55.105451 23376 raft_consensus.cc:348] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:50:55.105415 23377 raft_consensus.cc:348] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:50:55.105908 23376 raft_consensus.cc:374] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:55.105909 23377 raft_consensus.cc:374] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:55.106006 23376 raft_consensus.cc:729] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5870f0fca3f3461d896040b6b582c2c4, State: Initialized, Role: FOLLOWER
I20260809 22:50:55.106019 23377 raft_consensus.cc:729] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a89b0c768042da89619795a57bb0fa, State: Initialized, Role: FOLLOWER
I20260809 22:50:55.106778 23376 consensus_queue.cc:260] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:50:55.106796 23377 consensus_queue.cc:260] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:50:55.110453 23376 ts_tablet_manager.cc:1428] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260809 22:50:55.110527 23377 ts_tablet_manager.cc:1428] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: Time spent starting tablet: real 0.016s user 0.012s sys 0.001s
I20260809 22:50:55.111169 23097 heartbeater.cc:503] Master 127.24.136.62:42387 was elected leader, sending a full tablet report...
W20260809 22:50:55.125068 23098 tablet.cc:2369] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:50:55.193806 23340 tablet.cc:2369] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:50:55.222393 23386 raft_consensus.cc:482] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:50:55.222747 23386 raft_consensus.cc:504] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:50:55.224764 23386 leader_election.cc:288] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5870f0fca3f3461d896040b6b582c2c4 (127.24.136.2:43393), f6a89b0c768042da89619795a57bb0fa (127.24.136.1:39705)
I20260809 22:50:55.231678 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.225364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53319 (local address 127.24.136.2:43393)
0809 22:50:55.225589 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:55.225595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:55.225760 (+ 165us) server_negotiation.cc:407] Connection header received
0809 22:50:55.225969 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:55.225975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:55.226073 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:55.226189 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:55.226839 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.227700 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.229690 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.230720 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.230881 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.230961 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:55.231257 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 22:50:55.231394 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:50:55.231706 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.225396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:43393 (local address 127.24.136.3:53319)
0809 22:50:55.225614 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:55.225637 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:55.225794 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:55.226309 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:55.226315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:55.226352 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:55.226569 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.226583 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.227865 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.227869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.229468 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.229486 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.230871 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.230877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.231080 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.231108 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:50:55.231214 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 22:50:55.231371 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":1}
I20260809 22:50:55.232468 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.225443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:36853 (local address 127.24.136.1:39705)
0809 22:50:55.225666 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:55.225671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:55.226734 (+ 1063us) server_negotiation.cc:407] Connection header received
0809 22:50:55.226936 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:55.226941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:55.227007 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:55.227083 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:55.227696 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.228436 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.230598 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.231524 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.231691 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.231768 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:55.232080 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 22:50:55.232225 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 22:50:55.232554 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.225414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:39705 (local address 127.24.136.3:36853)
0809 22:50:55.226572 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:55.226593 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:55.226767 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:55.227129 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:55.227134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:55.227160 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:55.227509 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.227519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.228630 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.228634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.230412 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:55.230427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.231744 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.231751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:55.231910 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.231935 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:50:55.232017 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 22:50:55.232150 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"mutex_wait_us":213,"thread_start_us":488,"threads_started":1}
I20260809 22:50:55.233096 23016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5218aa3e11e469d8ccc32ed2a03a22e" candidate_uuid: "b797ffd76aa3443da6d166e7a38d992f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5870f0fca3f3461d896040b6b582c2c4" is_pre_election: true
I20260809 22:50:55.233770 23016 raft_consensus.cc:2454] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b797ffd76aa3443da6d166e7a38d992f in term 0.
I20260809 22:50:55.233713 22794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5218aa3e11e469d8ccc32ed2a03a22e" candidate_uuid: "b797ffd76aa3443da6d166e7a38d992f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a89b0c768042da89619795a57bb0fa" is_pre_election: true
I20260809 22:50:55.234323 22794 raft_consensus.cc:2454] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b797ffd76aa3443da6d166e7a38d992f in term 0.
I20260809 22:50:55.234611 23140 leader_election.cc:302] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5870f0fca3f3461d896040b6b582c2c4, b797ffd76aa3443da6d166e7a38d992f; no voters:
I20260809 22:50:55.235038 23386 raft_consensus.cc:2781] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:50:55.235201 23386 raft_consensus.cc:482] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:50:55.235304 23386 raft_consensus.cc:3037] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:55.237852 23386 raft_consensus.cc:504] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:50:55.238584 23386 leader_election.cc:288] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [CANDIDATE]: Term 1 election: Requested vote from peers 5870f0fca3f3461d896040b6b582c2c4 (127.24.136.2:43393), f6a89b0c768042da89619795a57bb0fa (127.24.136.1:39705)
I20260809 22:50:55.239156 23016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5218aa3e11e469d8ccc32ed2a03a22e" candidate_uuid: "b797ffd76aa3443da6d166e7a38d992f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5870f0fca3f3461d896040b6b582c2c4"
I20260809 22:50:55.239212 22794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5218aa3e11e469d8ccc32ed2a03a22e" candidate_uuid: "b797ffd76aa3443da6d166e7a38d992f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a89b0c768042da89619795a57bb0fa"
I20260809 22:50:55.239370 23016 raft_consensus.cc:3037] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:55.239423 22794 raft_consensus.cc:3037] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:55.241701 23016 raft_consensus.cc:2454] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b797ffd76aa3443da6d166e7a38d992f in term 1.
I20260809 22:50:55.241783 22794 raft_consensus.cc:2454] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b797ffd76aa3443da6d166e7a38d992f in term 1.
I20260809 22:50:55.242233 23140 leader_election.cc:302] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5870f0fca3f3461d896040b6b582c2c4, b797ffd76aa3443da6d166e7a38d992f; no voters:
I20260809 22:50:55.242576 23386 raft_consensus.cc:2781] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:50:55.243048 23386 raft_consensus.cc:686] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 1 LEADER]: Becoming Leader. State: Replica: b797ffd76aa3443da6d166e7a38d992f, State: Running, Role: LEADER
I20260809 22:50:55.243634 23386 consensus_queue.cc:237] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } }
I20260809 22:50:55.248425 22537 catalog_manager.cc:5495] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f reported cstate change: term changed from 0 to 1, leader changed from <none> to b797ffd76aa3443da6d166e7a38d992f (127.24.136.3). New cstate: current_term: 1 leader_uuid: "b797ffd76aa3443da6d166e7a38d992f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b797ffd76aa3443da6d166e7a38d992f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 33237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 } health_report { overall_health: UNKNOWN } } }
I20260809 22:50:55.290032 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.284459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.24.136.3:33237)
0809 22:50:55.284743 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:55.284747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:55.284768 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:50:55.284921 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:55.284925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:55.285010 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:55.285156 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:50:55.285547 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.286318 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.287833 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.288266 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.288377 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.288878 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:50:55.289421 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:50:55.289585 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:55.289640 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 22:50:55.289753 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260809 22:50:55.299646 23016 raft_consensus.cc:1261] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [term 1 FOLLOWER]: Refusing update from remote peer b797ffd76aa3443da6d166e7a38d992f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:50:55.299646 22794 raft_consensus.cc:1261] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [term 1 FOLLOWER]: Refusing update from remote peer b797ffd76aa3443da6d166e7a38d992f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:50:55.301054 23392 consensus_queue.cc:1035] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6a89b0c768042da89619795a57bb0fa" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 39705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:50:55.301510 22869 tablet.cc:2369] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:50:55.301641 23386 consensus_queue.cc:1035] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5870f0fca3f3461d896040b6b582c2c4" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 43393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:50:55.329672 22536 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:50:55.330060 22536 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:50:55.334683 22536 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33a93581187d47e0b9d41665cff134f2: Sending DeleteTablet for 3 replicas of tablet b5218aa3e11e469d8ccc32ed2a03a22e
I20260809 22:50:55.336162 22979 tablet_service.cc:1514] Processing DeleteTablet for tablet b5218aa3e11e469d8ccc32ed2a03a22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:50:55 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:43236
I20260809 22:50:55.336196 22757 tablet_service.cc:1514] Processing DeleteTablet for tablet b5218aa3e11e469d8ccc32ed2a03a22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:50:55 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:47456
I20260809 22:50:55.337055 23219 tablet_service.cc:1514] Processing DeleteTablet for tablet b5218aa3e11e469d8ccc32ed2a03a22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:50:55 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:34684
I20260809 22:50:55.337610 23420 tablet_replica.cc:330] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: stopping tablet replica
I20260809 22:50:55.337728 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 22640
I20260809 22:50:55.338115 23420 raft_consensus.cc:2229] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:50:55.338670 23420 raft_consensus.cc:2258] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:50:55.339564 23418 tablet_replica.cc:330] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: stopping tablet replica
I20260809 22:50:55.340066 23418 raft_consensus.cc:2229] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:50:55.340660 23418 raft_consensus.cc:2258] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:50:55.343403 23420 ts_tablet_manager.cc:1905] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:50:55.350438 23418 ts_tablet_manager.cc:1905] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:50:55.356148 23423 tablet_replica.cc:330] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: stopping tablet replica
I20260809 22:50:55.357167 23423 raft_consensus.cc:2229] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:50:55.357995 23423 raft_consensus.cc:2258] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 22:50:55 dist-test-slave-pww0 krb5kdc[22457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:50:55.360059 23420 ts_tablet_manager.cc:1918] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:50:55.360405 23420 log.cc:1196] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/b5218aa3e11e469d8ccc32ed2a03a22e
I20260809 22:50:55.361052 23420 ts_tablet_manager.cc:1939] T b5218aa3e11e469d8ccc32ed2a03a22e P 5870f0fca3f3461d896040b6b582c2c4: Deleting consensus metadata
I20260809 22:50:55.361505 23418 ts_tablet_manager.cc:1918] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:50:55.361734 23418 log.cc:1196] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/b5218aa3e11e469d8ccc32ed2a03a22e
I20260809 22:50:55.362766 23418 ts_tablet_manager.cc:1939] T b5218aa3e11e469d8ccc32ed2a03a22e P f6a89b0c768042da89619795a57bb0fa: Deleting consensus metadata
I20260809 22:50:55.363525 22509 catalog_manager.cc:4841] TS 5870f0fca3f3461d896040b6b582c2c4 (127.24.136.2:43393): tablet b5218aa3e11e469d8ccc32ed2a03a22e (table test-table [id=f54b43582b4d45318aa926ad9b4193f4]) successfully deleted
I20260809 22:50:55.365242 23423 ts_tablet_manager.cc:1905] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:50:55.365612 22513 catalog_manager.cc:4841] TS f6a89b0c768042da89619795a57bb0fa (127.24.136.1:39705): tablet b5218aa3e11e469d8ccc32ed2a03a22e (table test-table [id=f54b43582b4d45318aa926ad9b4193f4]) successfully deleted
I20260809 22:50:55.369762 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.340682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47462 (local address 127.24.136.1:39705)
0809 22:50:55.340894 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:55.340901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:55.348470 (+ 7569us) server_negotiation.cc:407] Connection header received
0809 22:50:55.348760 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:55.348766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:55.348862 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:55.348969 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:55.351202 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.352456 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.354482 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.355175 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.355293 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.361251 (+ 5958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:55.361274 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:55.361296 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:55.361349 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:55.365814 (+ 4465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:55.366837 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:55.366848 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:55.366851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:55.367029 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:55.367601 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:55.367607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:55.367610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:55.368116 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:55.368429 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:55.369092 (+ 663us) server_negotiation.cc:299] Negotiation successful
0809 22:50:55.369374 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:50:55.371297 23423 ts_tablet_manager.cc:1918] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:50:55.371750 22861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:55.371601 23423 log.cc:1196] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/b5218aa3e11e469d8ccc32ed2a03a22e
I20260809 22:50:55.372596 23423 ts_tablet_manager.cc:1939] T b5218aa3e11e469d8ccc32ed2a03a22e P b797ffd76aa3443da6d166e7a38d992f: Deleting consensus metadata
I20260809 22:50:55.374603 22509 catalog_manager.cc:4841] TS b797ffd76aa3443da6d166e7a38d992f (127.24.136.3:33237): tablet b5218aa3e11e469d8ccc32ed2a03a22e (table test-table [id=f54b43582b4d45318aa926ad9b4193f4]) successfully deleted
I20260809 22:50:55.474037 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 22640
I20260809 22:50:55.499792 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 22877
Aug 09 22:50:55 dist-test-slave-pww0 krb5kdc[22457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:50:55.521240 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.500622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43252 (local address 127.24.136.2:43393)
0809 22:50:55.500807 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:55.500813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:55.500897 (+ 84us) server_negotiation.cc:407] Connection header received
0809 22:50:55.501099 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:55.501104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:55.501211 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:55.501308 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:55.502636 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.503410 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.504855 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.505275 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.505395 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.514573 (+ 9178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:55.514591 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:55.514610 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:55.514657 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:55.517895 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:55.518786 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:55.518795 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:55.518798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:55.518949 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:55.519414 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:55.519419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:55.519421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:55.519928 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:55.520242 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:55.520602 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 22:50:55.520856 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:50:55.521931 23090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:55.670768 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 22877
I20260809 22:50:55.697820 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 23107
Aug 09 22:50:55 dist-test-slave-pww0 krb5kdc[22457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:50:55.715821 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:55.698714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.24.136.3:33237)
0809 22:50:55.699220 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:55.699224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:55.699241 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:50:55.699302 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:55.699306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:55.699395 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:55.699527 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:55.700933 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.701781 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.703129 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:55.703808 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:55.703949 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:55.708759 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:55.708777 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:55.708795 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:50:55.708844 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:55.712473 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:55.713324 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:55.713334 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:55.713336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:55.713542 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:50:55.714020 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:50:55.714024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:50:55.714026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:50:55.714442 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:55.714751 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:55.715152 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 22:50:55.715407 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380}
I20260809 22:50:55.716796 23329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:55.845929 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 23107
I20260809 22:50:55.871430 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 22481
I20260809 22:50:55.872848 22595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:55.987628 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 22481
2026-08-09T22:50:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5019 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T22:50:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:50:56Z Disabled control of system clock
I20260809 22:50:56.088966 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:43697
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:33611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:43697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:50:56.458427 23444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:56.458778 23444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:56.458899 23444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:56.466609 23444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:50:56.466748 23444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:56.466787 23444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:56.466845 23444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:50:56.466889 23444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:50:56.476639 23444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:43697
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:43697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:56.477957 23444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:56.479785 23444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:56.492136 23453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:56.492168 23455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:56.492201 23458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:56.493002 23444 server_base.cc:1034] running on GCE node
I20260809 22:50:56.493883 23444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:56.495667 23444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:56.496888 23444 hybrid_clock.cc:648] HybridClock initialized: now 1786315856496836 us; error 53 us; skew 500 ppm
I20260809 22:50:56.497334 23444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:50:56.499953 23444 webserver.cc:460] Webserver started at http://127.24.136.62:38215/ using document root <none> and password file <none>
I20260809 22:50:56.500734 23444 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:56.500837 23444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:56.501164 23444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:50:56.504477 23444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "0bd877a256d541d6ad488d382f6943a5"
format_stamp: "Formatted at 2026-08-09 22:50:56 on dist-test-slave-pww0"
server_key: "6e3c645ffd932c1806a60481ee8636e5"
server_key_iv: "59ccfdc6292e78a289539e5d89cbe211"
server_key_version: "encryptionkey@0"
I20260809 22:50:56.505394 23444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "0bd877a256d541d6ad488d382f6943a5"
format_stamp: "Formatted at 2026-08-09 22:50:56 on dist-test-slave-pww0"
server_key: "6e3c645ffd932c1806a60481ee8636e5"
server_key_iv: "59ccfdc6292e78a289539e5d89cbe211"
server_key_version: "encryptionkey@0"
I20260809 22:50:56.511327 23444 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 22:50:56.515403 23464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:56.517009 23444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:50:56.517207 23444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "0bd877a256d541d6ad488d382f6943a5"
format_stamp: "Formatted at 2026-08-09 22:50:56 on dist-test-slave-pww0"
server_key: "6e3c645ffd932c1806a60481ee8636e5"
server_key_iv: "59ccfdc6292e78a289539e5d89cbe211"
server_key_version: "encryptionkey@0"
I20260809 22:50:56.517452 23444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:56.537684 23444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:56.538677 23444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:56.538990 23444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:56.558089 23444 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:43697
I20260809 22:50:56.558157 23557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:43697 every 8 connection(s)
I20260809 22:50:56.559607 23444 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:50:56.563306 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 23444
I20260809 22:50:56.563540 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:50:56.564023 25120 external_mini_cluster.cc:1428] Setting key 44164e75d7b906322c8c2eabc4ac1ccf
I20260809 22:50:56.564023 23558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:56.579629 23558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5: Bootstrap starting.
I20260809 22:50:56.580641 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:56.566078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42656 (local address 127.24.136.62:43697)
0809 22:50:56.567224 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:56.567240 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:56.567292 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:50:56.569379 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:56.569408 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:56.570004 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:56.570481 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:56.571179 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:56.573188 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:56.575442 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:56.576335 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:56.576519 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:56.577933 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:56.577942 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:56.577960 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:50:56.578563 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:56.578723 (+ 160us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:50:56.578748 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:56.578861 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:56.579149 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 22:50:56.579541 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":423,"threads_started":1}
I20260809 22:50:56.583560 23558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:56.584707 23558 log.cc:824] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:56.587776 23558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5: No bootstrap required, opened a new log
I20260809 22:50:56.596848 23558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd877a256d541d6ad488d382f6943a5" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43697 } }
I20260809 22:50:56.597296 23558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:56.597400 23558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd877a256d541d6ad488d382f6943a5, State: Initialized, Role: FOLLOWER
I20260809 22:50:56.598065 23558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd877a256d541d6ad488d382f6943a5" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43697 } }
I20260809 22:50:56.598373 23558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:50:56.598523 23558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:50:56.598675 23558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:56.601187 23558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd877a256d541d6ad488d382f6943a5" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43697 } }
I20260809 22:50:56.601874 23558 leader_election.cc:302] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bd877a256d541d6ad488d382f6943a5; no voters:
I20260809 22:50:56.602635 23558 leader_election.cc:288] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:50:56.602746 23566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:50:56.603528 23566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [term 1 LEADER]: Becoming Leader. State: Replica: 0bd877a256d541d6ad488d382f6943a5, State: Running, Role: LEADER
I20260809 22:50:56.604104 23566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd877a256d541d6ad488d382f6943a5" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43697 } }
I20260809 22:50:56.604944 23558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:50:56.607177 23569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd877a256d541d6ad488d382f6943a5. Latest consensus state: current_term: 1 leader_uuid: "0bd877a256d541d6ad488d382f6943a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd877a256d541d6ad488d382f6943a5" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43697 } } }
I20260809 22:50:56.607743 23569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [sys.catalog]: This master's current role is: LEADER
I20260809 22:50:56.607739 23567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bd877a256d541d6ad488d382f6943a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd877a256d541d6ad488d382f6943a5" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43697 } } }
I20260809 22:50:56.608139 23567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5 [sys.catalog]: This master's current role is: LEADER
I20260809 22:50:56.610394 23574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:50:56.616084 23574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:50:56.626856 23574 catalog_manager.cc:1347] Generated new cluster ID: 084467841bc54f02bdc88aae1e01e97b
I20260809 22:50:56.626991 23574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:50:56.657698 23574 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:50:56.659737 23574 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:50:56.668887 23574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5: Generated new TSK 0
I20260809 22:50:56.670112 23574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:50:56.687217 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:43697
--builtin_ntp_servers=127.24.136.20:33611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:50:57.051366 23595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:57.051772 23595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:57.051872 23595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:57.059849 23595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:57.059973 23595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:57.060086 23595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:50:57.070084 23595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:43697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:57.071419 23595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:57.073280 23595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:57.087706 23607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:57.087755 23609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:57.088025 23605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:57.088585 23595 server_base.cc:1034] running on GCE node
I20260809 22:50:57.089262 23595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:57.090337 23595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:57.091634 23595 hybrid_clock.cc:648] HybridClock initialized: now 1786315857091562 us; error 67 us; skew 500 ppm
I20260809 22:50:57.092016 23595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:50:57.094703 23595 webserver.cc:460] Webserver started at http://127.24.136.1:37685/ using document root <none> and password file <none>
I20260809 22:50:57.095592 23595 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:57.095728 23595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:57.096094 23595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:50:57.099620 23595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "a56ed8ac2318496d95daa82d9b56cb44"
format_stamp: "Formatted at 2026-08-09 22:50:57 on dist-test-slave-pww0"
server_key: "8792a8f4b7134326b920ddf2db3efef6"
server_key_iv: "ba323be2e65b8a41f2dbaffbb4d3cc9c"
server_key_version: "encryptionkey@0"
I20260809 22:50:57.100559 23595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "a56ed8ac2318496d95daa82d9b56cb44"
format_stamp: "Formatted at 2026-08-09 22:50:57 on dist-test-slave-pww0"
server_key: "8792a8f4b7134326b920ddf2db3efef6"
server_key_iv: "ba323be2e65b8a41f2dbaffbb4d3cc9c"
server_key_version: "encryptionkey@0"
I20260809 22:50:57.106781 23595 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 22:50:57.111194 23617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:57.113211 23595 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:50:57.113457 23595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "a56ed8ac2318496d95daa82d9b56cb44"
format_stamp: "Formatted at 2026-08-09 22:50:57 on dist-test-slave-pww0"
server_key: "8792a8f4b7134326b920ddf2db3efef6"
server_key_iv: "ba323be2e65b8a41f2dbaffbb4d3cc9c"
server_key_version: "encryptionkey@0"
I20260809 22:50:57.113708 23595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:57.134007 23595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:57.134964 23595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:57.135298 23595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:57.136711 23595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:57.138475 23595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:57.138578 23595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:57.138691 23595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:57.138756 23595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:57.178165 23595 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:43347
I20260809 22:50:57.178179 23797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:43347 every 8 connection(s)
I20260809 22:50:57.179713 23595 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:50:57.182088 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 23595
I20260809 22:50:57.182260 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:50:57.182791 25120 external_mini_cluster.cc:1428] Setting key adb882de9d39690c930af7d8f114d4dc
I20260809 22:50:57.185808 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:43697
--builtin_ntp_servers=127.24.136.20:33611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:50:57.194294 23800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:57.182905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:34627 (local address 127.24.136.62:43697)
0809 22:50:57.183528 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:57.183534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:57.184743 (+ 1209us) server_negotiation.cc:407] Connection header received
0809 22:50:57.186738 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:57.186745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:57.186832 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:57.186952 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:57.188191 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:57.189189 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:57.190993 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:57.191603 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:57.191741 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:57.193324 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:57.193331 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:57.193335 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:50:57.193361 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:57.193415 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:50:57.193431 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:57.193524 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:57.193829 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 22:50:57.193944 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":297,"threads_started":1}
I20260809 22:50:57.195412 23801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:57.183350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:43697 (local address 127.24.136.1:34627)
0809 22:50:57.184570 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:57.184628 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:57.186420 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:57.187168 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:57.187176 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:57.187294 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:57.187994 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:57.188014 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:57.189401 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:57.189406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:57.190784 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:57.190800 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:57.191869 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:57.191875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:57.192113 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:57.192905 (+ 792us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:50:57.192908 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:57.192956 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:50:57.193056 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:57.193576 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:57.193591 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:57.193599 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:50:57.193786 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 22:50:57.194187 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":349,"threads_started":1}
I20260809 22:50:57.197373 23798 heartbeater.cc:347] Connected to a master server at 127.24.136.62:43697
I20260809 22:50:57.197710 23798 heartbeater.cc:464] Registering TS with master...
I20260809 22:50:57.198664 23798 heartbeater.cc:511] Master 127.24.136.62:43697 requested a full tablet report, sending...
I20260809 22:50:57.201468 23496 ts_manager.cc:194] Registered new tserver with Master: a56ed8ac2318496d95daa82d9b56cb44 (127.24.136.1:43347)
I20260809 22:50:57.203563 23496 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.1:34627
W20260809 22:50:57.545117 23802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:57.545437 23802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:57.545526 23802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:57.553061 23802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:57.553184 23802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:57.553294 23802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:50:57.562880 23802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:43697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:57.564332 23802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:57.566243 23802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:57.580669 23813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:57.580709 23816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:57.580608 23820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:57.582360 23802 server_base.cc:1034] running on GCE node
I20260809 22:50:57.583144 23802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:57.584398 23802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:57.585649 23802 hybrid_clock.cc:648] HybridClock initialized: now 1786315857585577 us; error 62 us; skew 500 ppm
I20260809 22:50:57.586002 23802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:50:57.588747 23802 webserver.cc:460] Webserver started at http://127.24.136.2:36385/ using document root <none> and password file <none>
I20260809 22:50:57.589583 23802 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:57.589715 23802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:57.590085 23802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:50:57.594848 23802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "5cd3352a5106479480e3150416514448"
format_stamp: "Formatted at 2026-08-09 22:50:57 on dist-test-slave-pww0"
server_key: "28ed5c306e570d42c2e5452e7124e5ab"
server_key_iv: "86cfc455c7f0354ea159cfd1eabbc871"
server_key_version: "encryptionkey@0"
I20260809 22:50:57.596001 23802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "5cd3352a5106479480e3150416514448"
format_stamp: "Formatted at 2026-08-09 22:50:57 on dist-test-slave-pww0"
server_key: "28ed5c306e570d42c2e5452e7124e5ab"
server_key_iv: "86cfc455c7f0354ea159cfd1eabbc871"
server_key_version: "encryptionkey@0"
I20260809 22:50:57.602600 23802 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 22:50:57.607000 23836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:57.608775 23802 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:50:57.608985 23802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "5cd3352a5106479480e3150416514448"
format_stamp: "Formatted at 2026-08-09 22:50:57 on dist-test-slave-pww0"
server_key: "28ed5c306e570d42c2e5452e7124e5ab"
server_key_iv: "86cfc455c7f0354ea159cfd1eabbc871"
server_key_version: "encryptionkey@0"
I20260809 22:50:57.609233 23802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:57.639505 23802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:57.640518 23802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:57.640868 23802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:57.642047 23802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:57.643846 23802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:57.643959 23802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:57.644085 23802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:57.644157 23802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:57.677904 23802 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:41459
I20260809 22:50:57.677968 24026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:41459 every 8 connection(s)
I20260809 22:50:57.679358 23802 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:50:57.683068 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 23802
I20260809 22:50:57.683271 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:50:57.683785 25120 external_mini_cluster.cc:1428] Setting key 02c7761a447d2768e8cf6f045b0ecf81
I20260809 22:50:57.687002 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:43697
--builtin_ntp_servers=127.24.136.20:33611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:50:57.693583 23800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:57.682335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:56819 (local address 127.24.136.62:43697)
0809 22:50:57.682539 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:57.682543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:57.684218 (+ 1675us) server_negotiation.cc:407] Connection header received
0809 22:50:57.686270 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:57.686276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:57.686367 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:57.686480 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:57.687731 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:57.688811 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:57.690602 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:57.691136 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:57.691266 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:57.692753 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:57.692760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:57.692764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:50:57.692791 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:57.692831 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:50:57.692846 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:57.692934 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:57.693157 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 22:50:57.693274 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:50:57.694643 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:57.682819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:43697 (local address 127.24.136.2:56819)
0809 22:50:57.683996 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:57.684052 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:57.685956 (+ 1904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:57.686690 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:57.686699 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:57.686821 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:57.687535 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:57.687564 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:57.689010 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:57.689015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:57.690421 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:57.690435 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:57.691359 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:57.691364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:57.691648 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:57.692379 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:50:57.692381 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:57.692434 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:50:57.692529 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:57.692935 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:57.692942 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:57.692948 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:50:57.693102 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 22:50:57.693476 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":361,"threads_started":1}
I20260809 22:50:57.696338 24029 heartbeater.cc:347] Connected to a master server at 127.24.136.62:43697
I20260809 22:50:57.696714 24029 heartbeater.cc:464] Registering TS with master...
I20260809 22:50:57.697708 24029 heartbeater.cc:511] Master 127.24.136.62:43697 requested a full tablet report, sending...
I20260809 22:50:57.699585 23496 ts_manager.cc:194] Registered new tserver with Master: 5cd3352a5106479480e3150416514448 (127.24.136.2:41459)
I20260809 22:50:57.700523 23496 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.2:56819
W20260809 22:50:58.044008 24034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:50:58.044341 24034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:50:58.044461 24034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:50:58.052070 24034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:50:58.052193 24034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:50:58.052340 24034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:50:58.061985 24034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:43697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:50:58.063410 24034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:50:58.065382 24034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:50:58.079389 24048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:58.079434 24046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:50:58.079491 24052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:50:58.080977 24034 server_base.cc:1034] running on GCE node
I20260809 22:50:58.081717 24034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:50:58.082831 24034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:50:58.084084 24034 hybrid_clock.cc:648] HybridClock initialized: now 1786315858084017 us; error 60 us; skew 500 ppm
I20260809 22:50:58.084450 24034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:50:58.086988 24034 webserver.cc:460] Webserver started at http://127.24.136.3:40745/ using document root <none> and password file <none>
I20260809 22:50:58.087806 24034 fs_manager.cc:359] Metadata directory not provided
I20260809 22:50:58.087909 24034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:50:58.088228 24034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:50:58.091354 24034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "3d6b4e66487f4787823df9e135e3f7e5"
format_stamp: "Formatted at 2026-08-09 22:50:58 on dist-test-slave-pww0"
server_key: "35aff8822cc1fcb4a21961ededdf30e0"
server_key_iv: "a288950d9d77dd9197f8e66c6238d47f"
server_key_version: "encryptionkey@0"
I20260809 22:50:58.092272 24034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "3d6b4e66487f4787823df9e135e3f7e5"
format_stamp: "Formatted at 2026-08-09 22:50:58 on dist-test-slave-pww0"
server_key: "35aff8822cc1fcb4a21961ededdf30e0"
server_key_iv: "a288950d9d77dd9197f8e66c6238d47f"
server_key_version: "encryptionkey@0"
I20260809 22:50:58.098070 24034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:50:58.102347 24065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:58.103899 24034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:50:58.104079 24034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "3d6b4e66487f4787823df9e135e3f7e5"
format_stamp: "Formatted at 2026-08-09 22:50:58 on dist-test-slave-pww0"
server_key: "35aff8822cc1fcb4a21961ededdf30e0"
server_key_iv: "a288950d9d77dd9197f8e66c6238d47f"
server_key_version: "encryptionkey@0"
I20260809 22:50:58.104310 24034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:50:58.133701 24034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:50:58.134651 24034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:50:58.134958 24034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:50:58.136139 24034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:50:58.137876 24034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:50:58.137972 24034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:58.138087 24034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:50:58.138154 24034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:50:58.172596 24034 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:40755
I20260809 22:50:58.172665 24263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:40755 every 8 connection(s)
I20260809 22:50:58.174014 24034 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:50:58.183964 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 24034
I20260809 22:50:58.184139 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:50:58.184573 25120 external_mini_cluster.cc:1428] Setting key 1f85d2a806ebd69e88334bc7c7f51aca
I20260809 22:50:58.188761 23800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.176908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:54175 (local address 127.24.136.62:43697)
0809 22:50:58.177098 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.177103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.178975 (+ 1872us) server_negotiation.cc:407] Connection header received
0809 22:50:58.180963 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.180970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.181074 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.181194 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:58.182841 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.183729 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.185634 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.186198 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.186326 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.188053 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:58.188058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:58.188061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:50:58.188080 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:58.188113 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:50:58.188125 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:58.188196 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.188418 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.188510 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:50:58.189910 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.177477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:43697 (local address 127.24.136.3:54175)
0809 22:50:58.178644 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.178772 (+ 128us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:58.180604 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:58.181537 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:58.181550 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:58.181686 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:50:58.182587 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.182617 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.183912 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.183917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.185448 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.185465 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.186450 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.186455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.186815 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.187674 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:50:58.187676 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:50:58.187732 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:50:58.187834 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:50:58.188212 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:50:58.188221 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:50:58.188227 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:50:58.188394 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 22:50:58.188762 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":377,"threads_started":1}
I20260809 22:50:58.191563 24264 heartbeater.cc:347] Connected to a master server at 127.24.136.62:43697
I20260809 22:50:58.191890 24264 heartbeater.cc:464] Registering TS with master...
I20260809 22:50:58.192737 24264 heartbeater.cc:511] Master 127.24.136.62:43697 requested a full tablet report, sending...
I20260809 22:50:58.194339 23496 ts_manager.cc:194] Registered new tserver with Master: 3d6b4e66487f4787823df9e135e3f7e5 (127.24.136.3:40755)
I20260809 22:50:58.195153 23496 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.3:54175
I20260809 22:50:58.201007 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:50:58.207758 23798 heartbeater.cc:503] Master 127.24.136.62:43697 was elected leader, sending a full tablet report...
I20260809 22:50:58.212313 23800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.205479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42658 (local address 127.24.136.62:43697)
0809 22:50:58.205805 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.205812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.206259 (+ 447us) server_negotiation.cc:407] Connection header received
0809 22:50:58.206534 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.206539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.206612 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.206714 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:58.207261 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.208095 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.209853 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.210310 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.210428 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.211584 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:58.211589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:58.211593 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:50:58.211613 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:58.211649 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:50:58.211662 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:58.211741 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.211898 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.212000 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260809 22:50:58.216679 23496 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:50:58.224012 23496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:50:58.248660 24285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.237912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:41459 (local address 127.0.0.1:51114)
0809 22:50:58.238940 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.239000 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:58.239244 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:58.240791 (+ 1547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:58.240798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:58.240849 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:58.241377 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.241397 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.243232 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.243237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.245823 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.245838 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.247259 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.247265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.247805 (+ 540us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.247871 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 22:50:58.248000 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 22:50:58.248213 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":170,"spinlock_wait_cycles":14080,"thread_start_us":506,"threads_started":1}
I20260809 22:50:58.249503 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.238066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51114 (local address 127.24.136.2:41459)
0809 22:50:58.239851 (+ 1785us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.239859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.239891 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:50:58.239998 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.240005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.240282 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.240496 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:58.241604 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.243092 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.246059 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.247060 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.247209 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.247525 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.248956 (+ 1431us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.249156 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1646,"thread_start_us":1540,"threads_started":1}
I20260809 22:50:58.249526 24292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.239912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:40755 (local address 127.0.0.1:44166)
0809 22:50:58.240711 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.240729 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:58.240836 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:58.241987 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:58.241992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:58.242020 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:58.242402 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.242415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.244693 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.244698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.246905 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.246921 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.248752 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.248758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.248943 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.248978 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:50:58.249045 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 22:50:58.249169 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":52,"thread_start_us":878,"threads_started":1}
I20260809 22:50:58.250686 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.240530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.24.136.3:40755)
0809 22:50:58.241355 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.241374 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.241409 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:50:58.241513 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.241520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.241740 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.241951 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:58.242622 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.244517 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.247133 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.248523 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.249762 (+ 1239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.250054 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.250128 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.250325 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":782,"threads_started":1}
I20260809 22:50:58.253857 23907 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecd4d03b3244afd8513291df72a2da0 (DEFAULT_TABLE table=test-table [id=f5ea4143d4664f4aa43036ede8d27f93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:50:58.255573 23907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecd4d03b3244afd8513291df72a2da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:58.260170 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.238542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.24.136.1:43347)
0809 22:50:58.240184 (+ 1642us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.240193 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.240221 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:50:58.240336 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.240344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.240568 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.240768 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:58.241662 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.243142 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.246144 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.256924 (+ 10780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.257099 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.257408 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.259598 (+ 2190us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.259818 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1461,"thread_start_us":1640,"threads_started":1}
I20260809 22:50:58.259733 24147 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecd4d03b3244afd8513291df72a2da0 (DEFAULT_TABLE table=test-table [id=f5ea4143d4664f4aa43036ede8d27f93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:50:58.260977 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.238010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:43347 (local address 127.0.0.1:37564)
0809 22:50:58.239563 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.239579 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:58.239701 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:58.241010 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:58.241015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:58.241044 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:58.241485 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.241500 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.243282 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.243287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.245884 (+ 2597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.245904 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.259015 (+ 13111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.259023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.259229 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.259270 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:50:58.259376 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 22:50:58.260594 (+ 1218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1401,"mutex_wait_us":204,"thread_start_us":1268,"threads_started":1}
I20260809 22:50:58.261463 24147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecd4d03b3244afd8513291df72a2da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:58.264472 23694 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecd4d03b3244afd8513291df72a2da0 (DEFAULT_TABLE table=test-table [id=f5ea4143d4664f4aa43036ede8d27f93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:50:58.266032 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecd4d03b3244afd8513291df72a2da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:50:58.270910 24294 tablet_bootstrap.cc:492] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: Bootstrap starting.
I20260809 22:50:58.275900 24295 tablet_bootstrap.cc:492] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: Bootstrap starting.
I20260809 22:50:58.277370 24294 tablet_bootstrap.cc:654] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:58.278940 24294 log.cc:824] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:58.279417 24295 tablet_bootstrap.cc:654] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:58.280786 24295 log.cc:824] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:58.281716 24296 tablet_bootstrap.cc:492] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44: Bootstrap starting.
I20260809 22:50:58.283699 24294 tablet_bootstrap.cc:492] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: No bootstrap required, opened a new log
I20260809 22:50:58.283982 24294 ts_tablet_manager.cc:1397] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260809 22:50:58.284497 24295 tablet_bootstrap.cc:492] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: No bootstrap required, opened a new log
I20260809 22:50:58.284801 24295 ts_tablet_manager.cc:1397] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:50:58.287127 24296 tablet_bootstrap.cc:654] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44: Neither blocks nor log segments found. Creating new log.
I20260809 22:50:58.288581 24296 log.cc:824] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44: Log is configured to *not* fsync() on all Append() calls
I20260809 22:50:58.292387 24294 raft_consensus.cc:348] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } }
I20260809 22:50:58.292801 24294 raft_consensus.cc:374] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:58.292910 24294 raft_consensus.cc:729] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd3352a5106479480e3150416514448, State: Initialized, Role: FOLLOWER
I20260809 22:50:58.293498 24296 tablet_bootstrap.cc:492] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44: No bootstrap required, opened a new log
I20260809 22:50:58.293545 24294 consensus_queue.cc:260] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } }
I20260809 22:50:58.293828 24296 ts_tablet_manager.cc:1397] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260809 22:50:58.294785 24029 heartbeater.cc:503] Master 127.24.136.62:43697 was elected leader, sending a full tablet report...
I20260809 22:50:58.295392 24294 ts_tablet_manager.cc:1428] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260809 22:50:58.296003 24295 raft_consensus.cc:348] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } }
I20260809 22:50:58.296499 24295 raft_consensus.cc:374] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:58.296604 24295 raft_consensus.cc:729] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6b4e66487f4787823df9e135e3f7e5, State: Initialized, Role: FOLLOWER
I20260809 22:50:58.297345 24295 consensus_queue.cc:260] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } }
I20260809 22:50:58.298588 24264 heartbeater.cc:503] Master 127.24.136.62:43697 was elected leader, sending a full tablet report...
I20260809 22:50:58.299360 24295 ts_tablet_manager.cc:1428] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 22:50:58.304106 24296 raft_consensus.cc:348] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } }
I20260809 22:50:58.304525 24296 raft_consensus.cc:374] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:50:58.304628 24296 raft_consensus.cc:729] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a56ed8ac2318496d95daa82d9b56cb44, State: Initialized, Role: FOLLOWER
I20260809 22:50:58.305253 24296 consensus_queue.cc:260] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } }
I20260809 22:50:58.307050 24296 ts_tablet_manager.cc:1428] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 22:50:58.391785 24312 raft_consensus.cc:482] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:50:58.392136 24312 raft_consensus.cc:504] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } }
I20260809 22:50:58.393949 24312 leader_election.cc:288] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd3352a5106479480e3150416514448 (127.24.136.2:41459), 3d6b4e66487f4787823df9e135e3f7e5 (127.24.136.3:40755)
I20260809 22:50:58.400784 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.394531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:54023 (local address 127.24.136.3:40755)
0809 22:50:58.394789 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.394796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.395376 (+ 580us) server_negotiation.cc:407] Connection header received
0809 22:50:58.395602 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.395606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.395670 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.395747 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:58.396329 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.397027 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.398908 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.399782 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.399941 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.400020 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.400328 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.400545 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 22:50:58.400826 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.394588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:33685 (local address 127.24.136.2:41459)
0809 22:50:58.395030 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.395037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.395658 (+ 621us) server_negotiation.cc:407] Connection header received
0809 22:50:58.395843 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.395847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.395919 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.395996 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:50:58.396615 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.397295 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.399036 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.399812 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.399954 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.400028 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.400362 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.400494 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260809 22:50:58.401044 24316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.394429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:41459 (local address 127.24.136.1:33685)
0809 22:50:58.395545 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.395560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:58.395710 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:58.396050 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:58.396054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:58.396083 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:58.396458 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.396468 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.397440 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.397444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.398851 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.398863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.400019 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.400025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.400204 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.400229 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:50:58.400307 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 22:50:58.400467 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":70,"thread_start_us":760,"threads_started":1}
I20260809 22:50:58.401224 24314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.394273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:40755 (local address 127.24.136.1:54023)
0809 22:50:58.395236 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.395256 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:50:58.395432 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:50:58.395802 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:50:58.395807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:50:58.395834 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:50:58.396163 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.396173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.397171 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.397177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.398690 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:50:58.398703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.399975 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.399981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:50:58.400144 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.400188 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 22:50:58.400271 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 22:50:58.400423 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":25,"thread_start_us":644,"threads_started":1}
I20260809 22:50:58.402328 24183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecd4d03b3244afd8513291df72a2da0" candidate_uuid: "a56ed8ac2318496d95daa82d9b56cb44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d6b4e66487f4787823df9e135e3f7e5" is_pre_election: true
I20260809 22:50:58.402315 23952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecd4d03b3244afd8513291df72a2da0" candidate_uuid: "a56ed8ac2318496d95daa82d9b56cb44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd3352a5106479480e3150416514448" is_pre_election: true
I20260809 22:50:58.402957 23952 raft_consensus.cc:2454] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a56ed8ac2318496d95daa82d9b56cb44 in term 0.
I20260809 22:50:58.402957 24183 raft_consensus.cc:2454] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a56ed8ac2318496d95daa82d9b56cb44 in term 0.
I20260809 22:50:58.403766 23622 leader_election.cc:302] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d6b4e66487f4787823df9e135e3f7e5, a56ed8ac2318496d95daa82d9b56cb44; no voters:
I20260809 22:50:58.404165 24312 raft_consensus.cc:2781] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:50:58.404328 24312 raft_consensus.cc:482] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:50:58.404435 24312 raft_consensus.cc:3037] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:58.406806 24312 raft_consensus.cc:504] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } }
I20260809 22:50:58.407416 24312 leader_election.cc:288] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [CANDIDATE]: Term 1 election: Requested vote from peers 5cd3352a5106479480e3150416514448 (127.24.136.2:41459), 3d6b4e66487f4787823df9e135e3f7e5 (127.24.136.3:40755)
I20260809 22:50:58.407969 23952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecd4d03b3244afd8513291df72a2da0" candidate_uuid: "a56ed8ac2318496d95daa82d9b56cb44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd3352a5106479480e3150416514448"
I20260809 22:50:58.407965 24183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecd4d03b3244afd8513291df72a2da0" candidate_uuid: "a56ed8ac2318496d95daa82d9b56cb44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d6b4e66487f4787823df9e135e3f7e5"
I20260809 22:50:58.408181 24183 raft_consensus.cc:3037] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:58.408185 23952 raft_consensus.cc:3037] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:50:58.410583 23952 raft_consensus.cc:2454] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a56ed8ac2318496d95daa82d9b56cb44 in term 1.
I20260809 22:50:58.410712 24183 raft_consensus.cc:2454] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a56ed8ac2318496d95daa82d9b56cb44 in term 1.
I20260809 22:50:58.411103 23621 leader_election.cc:302] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd3352a5106479480e3150416514448, a56ed8ac2318496d95daa82d9b56cb44; no voters:
I20260809 22:50:58.411502 24312 raft_consensus.cc:2781] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:50:58.411976 24312 raft_consensus.cc:686] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 1 LEADER]: Becoming Leader. State: Replica: a56ed8ac2318496d95daa82d9b56cb44, State: Running, Role: LEADER
I20260809 22:50:58.412519 24312 consensus_queue.cc:237] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } }
I20260809 22:50:58.417640 23494 catalog_manager.cc:5495] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 reported cstate change: term changed from 0 to 1, leader changed from <none> to a56ed8ac2318496d95daa82d9b56cb44 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "a56ed8ac2318496d95daa82d9b56cb44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a56ed8ac2318496d95daa82d9b56cb44" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 } health_report { overall_health: UNKNOWN } } }
W20260809 22:50:58.425796 24265 tablet.cc:2369] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:50:58.431586 24030 tablet.cc:2369] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:50:58.432286 23799 tablet.cc:2369] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:50:58.454952 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.449720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.24.136.1:43347)
0809 22:50:58.449896 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.449900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.449917 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:50:58.450008 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.450012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.450129 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.450256 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:50:58.450657 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.451361 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.452755 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.453172 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.453268 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.453806 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:50:58.454354 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:50:58.454478 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.454667 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.454745 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:50:58.464008 24183 raft_consensus.cc:1261] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [term 1 FOLLOWER]: Refusing update from remote peer a56ed8ac2318496d95daa82d9b56cb44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:50:58.464007 23952 raft_consensus.cc:1261] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [term 1 FOLLOWER]: Refusing update from remote peer a56ed8ac2318496d95daa82d9b56cb44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:50:58.465082 24317 consensus_queue.cc:1035] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d6b4e66487f4787823df9e135e3f7e5" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:50:58.465474 24312 consensus_queue.cc:1035] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd3352a5106479480e3150416514448" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 41459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:50:58.491329 23496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:50:58.491708 23496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:50:58.496052 23496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bd877a256d541d6ad488d382f6943a5: Sending DeleteTablet for 3 replicas of tablet 5ecd4d03b3244afd8513291df72a2da0
I20260809 22:50:58.497498 24147 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecd4d03b3244afd8513291df72a2da0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:50:58 UTC) from {username='slave'} at 127.0.0.1:44166
I20260809 22:50:58.497464 23907 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecd4d03b3244afd8513291df72a2da0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:50:58 UTC) from {username='slave'} at 127.0.0.1:51114
I20260809 22:50:58.498183 23694 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecd4d03b3244afd8513291df72a2da0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:50:58 UTC) from {username='slave'} at 127.0.0.1:37564
I20260809 22:50:58.499388 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 23595
I20260809 22:50:58.506558 24344 tablet_replica.cc:330] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44: stopping tablet replica
I20260809 22:50:58.507200 24344 raft_consensus.cc:2229] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:50:58.507511 24343 tablet_replica.cc:330] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: stopping tablet replica
I20260809 22:50:58.507951 24343 raft_consensus.cc:2229] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:50:58.508764 24344 raft_consensus.cc:2258] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:50:58.509049 24342 tablet_replica.cc:330] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: stopping tablet replica
I20260809 22:50:58.509469 24342 raft_consensus.cc:2229] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:50:58.510035 24342 raft_consensus.cc:2258] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:50:58.510459 24343 raft_consensus.cc:2258] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:50:58.511833 24344 ts_tablet_manager.cc:1905] T 5ecd4d03b3244afd8513291df72a2da0 P a56ed8ac2318496d95daa82d9b56cb44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:50:58.512847 24342 ts_tablet_manager.cc:1905] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:50:58.516578 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.500835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.24.136.1:43347)
0809 22:50:58.501040 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.501047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.506816 (+ 5769us) server_negotiation.cc:407] Connection header received
0809 22:50:58.507017 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.507024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.507151 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.507272 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:58.508063 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.509324 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.511443 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.512142 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.512277 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.514239 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:58.514256 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:58.514279 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:50:58.515124 (+ 845us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:58.515188 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:50:58.515216 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:58.515303 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.516044 (+ 741us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.516161 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:50:58.517313 23793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:58.517993 24343 ts_tablet_manager.cc:1905] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:50:58.522689 24342 ts_tablet_manager.cc:1918] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:50:58.522894 24342 log.cc:1196] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/5ecd4d03b3244afd8513291df72a2da0
I20260809 22:50:58.523475 24342 ts_tablet_manager.cc:1939] T 5ecd4d03b3244afd8513291df72a2da0 P 3d6b4e66487f4787823df9e135e3f7e5: Deleting consensus metadata
I20260809 22:50:58.524503 24343 ts_tablet_manager.cc:1918] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:50:58.524672 24343 log.cc:1196] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/5ecd4d03b3244afd8513291df72a2da0
I20260809 22:50:58.525151 24343 ts_tablet_manager.cc:1939] T 5ecd4d03b3244afd8513291df72a2da0 P 5cd3352a5106479480e3150416514448: Deleting consensus metadata
I20260809 22:50:58.525640 23470 catalog_manager.cc:4841] TS 3d6b4e66487f4787823df9e135e3f7e5 (127.24.136.3:40755): tablet 5ecd4d03b3244afd8513291df72a2da0 (table test-table [id=f5ea4143d4664f4aa43036ede8d27f93]) successfully deleted
I20260809 22:50:58.526823 23468 catalog_manager.cc:4841] TS 5cd3352a5106479480e3150416514448 (127.24.136.2:41459): tablet 5ecd4d03b3244afd8513291df72a2da0 (table test-table [id=f5ea4143d4664f4aa43036ede8d27f93]) successfully deleted
I20260809 22:50:58.620781 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 23595
I20260809 22:50:58.644840 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 23802
W20260809 22:50:58.644865 23470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.136.1:43347 (error 108)
I20260809 22:50:58.646729 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.646302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:43347 (local address 127.0.0.1:37580)
0809 22:50:58.646476 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.646585 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 22:50:58.646984 23470 catalog_manager.cc:4584] TS a56ed8ac2318496d95daa82d9b56cb44 (127.24.136.1:43347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ecd4d03b3244afd8513291df72a2da0: Network error: Client connection negotiation failed: client connection to 127.24.136.1:43347: connect: Connection refused (error 111)
I20260809 22:50:58.652267 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.645668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51130 (local address 127.24.136.2:41459)
0809 22:50:58.645856 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.645862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.645971 (+ 109us) server_negotiation.cc:407] Connection header received
0809 22:50:58.646161 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.646167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.646258 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.646348 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:58.646814 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.647717 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.649119 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.649537 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.649664 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.650817 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:58.650827 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:58.650842 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:50:58.651435 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:58.651533 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:50:58.651559 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:58.651666 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.651819 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.651941 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:50:58.653023 24020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:58.700616 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.700129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:43347 (local address 127.0.0.1:37582)
0809 22:50:58.700355 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.700448 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 22:50:58.701886 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.701401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:43347 (local address 127.0.0.1:37590)
0809 22:50:58.701623 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.701705 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 22:50:58.743574 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 23802
I20260809 22:50:58.762691 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.762064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:43347 (local address 127.0.0.1:37596)
0809 22:50:58.762358 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.762495 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260809 22:50:58.764010 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.763532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:43347 (local address 127.0.0.1:37610)
0809 22:50:58.763696 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.763813 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 22:50:58.772984 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 24034
I20260809 22:50:58.779781 24293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.773812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44180 (local address 127.24.136.3:40755)
0809 22:50:58.774024 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:50:58.774029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:50:58.774050 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:50:58.774159 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:50:58.774163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:50:58.774270 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:50:58.774375 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:50:58.774750 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.775557 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.776899 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:50:58.777298 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:50:58.777395 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:50:58.778418 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:50:58.778428 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:50:58.778445 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:50:58.779027 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:50:58.779082 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:50:58.779104 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:50:58.779195 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 22:50:58.779356 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 22:50:58.779484 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 22:50:58.780462 24252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:58.845016 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.844384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:43347 (local address 127.0.0.1:37622)
0809 22:50:58.844677 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.844798 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260809 22:50:58.846311 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:50:58.845831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:43347 (local address 127.0.0.1:37636)
0809 22:50:58.846056 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 22:50:58.846151 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:43347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 22:50:58.870601 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 24034
I20260809 22:50:58.892798 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 23444
I20260809 22:50:58.893931 23550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:50:59.005028 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 23444
2026-08-09T22:50:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3016 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:50:59 dist-test-slave-pww0 krb5kdc[24361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:50:59 dist-test-slave-pww0 krb5kdc[24361](info): set up 2 sockets
Aug 09 22:50:59 dist-test-slave-pww0 krb5kdc[24361](info): commencing operation
krb5kdc: starting...
W20260809 22:51:00.305258 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:00 dist-test-slave-pww0 krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:51:00.635771 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:37919
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:37919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:01.003342 24381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:01.003754 24381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:01.003894 24381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:01.012001 24381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:01.012153 24381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:01.012216 24381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:01.012311 24381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:01.012364 24381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:01.023028 24381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:37919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:37919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:01.024528 24381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:01.025983 24381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:01.038651 24389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:01.038740 24392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:01.038695 24398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:01.039897 24381 server_base.cc:1034] running on GCE node
I20260809 22:51:01.040776 24381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:01.042366 24381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:01.043571 24381 hybrid_clock.cc:648] HybridClock initialized: now 1786315861043497 us; error 59 us; skew 500 ppm
I20260809 22:51:01.043980 24381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:01 dist-test-slave-pww0 krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315861, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:01.052767 24381 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:51:01.054544 24381 webserver.cc:460] Webserver started at http://127.24.136.62:45697/ using document root <none> and password file <none>
I20260809 22:51:01.055290 24381 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:01.055419 24381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:01.055784 24381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:01.059008 24381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "97fe7043c2184abd9b6ee25d9534b98d"
format_stamp: "Formatted at 2026-08-09 22:51:01 on dist-test-slave-pww0"
server_key: "2c1feec1e2ec6e5a9ed44ac0ed54652e"
server_key_iv: "87e603d40e020b357eee80fd6edd5a49"
server_key_version: "encryptionkey@0"
I20260809 22:51:01.059885 24381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "97fe7043c2184abd9b6ee25d9534b98d"
format_stamp: "Formatted at 2026-08-09 22:51:01 on dist-test-slave-pww0"
server_key: "2c1feec1e2ec6e5a9ed44ac0ed54652e"
server_key_iv: "87e603d40e020b357eee80fd6edd5a49"
server_key_version: "encryptionkey@0"
I20260809 22:51:01.065469 24381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:51:01.069408 24414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:01.070838 24381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:51:01.071026 24381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "97fe7043c2184abd9b6ee25d9534b98d"
format_stamp: "Formatted at 2026-08-09 22:51:01 on dist-test-slave-pww0"
server_key: "2c1feec1e2ec6e5a9ed44ac0ed54652e"
server_key_iv: "87e603d40e020b357eee80fd6edd5a49"
server_key_version: "encryptionkey@0"
I20260809 22:51:01.071240 24381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:01.095985 24381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:01.100895 24381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:01.101197 24381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:01.120549 24381 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:37919
I20260809 22:51:01.120551 24500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:37919 every 8 connection(s)
I20260809 22:51:01.122049 24381 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:01.126542 24501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:01.129356 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 24381
I20260809 22:51:01.129567 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:01.130185 25120 external_mini_cluster.cc:1428] Setting key 0635c4ebc8c64470b4fe60eac77e4f04
I20260809 22:51:01.141580 24501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d: Bootstrap starting.
I20260809 22:51:01.146158 24501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:01.147827 24501 log.cc:824] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:51:01 dist-test-slave-pww0 krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:01.151865 24501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d: No bootstrap required, opened a new log
I20260809 22:51:01.158396 24505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:01.133341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.24.136.62:37919)
0809 22:51:01.134470 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:01.134483 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:01.134530 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:51:01.136391 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:01.136419 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:01.136904 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:01.137293 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:01.139522 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:01.140929 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:01.143005 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:01.143880 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:01.144036 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:01.150102 (+ 6066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:01.150112 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:01.150132 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:01.150181 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:01.153460 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:01.154339 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:01.154347 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:01.154355 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:01.154558 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:01.155018 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:01.155022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:01.155025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:01.155757 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:01.156077 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:01.156636 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 22:51:01.157177 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":516,"threads_started":1}
I20260809 22:51:01.161662 24501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fe7043c2184abd9b6ee25d9534b98d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37919 } }
I20260809 22:51:01.162233 24501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:01.162374 24501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97fe7043c2184abd9b6ee25d9534b98d, State: Initialized, Role: FOLLOWER
I20260809 22:51:01.163008 24501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fe7043c2184abd9b6ee25d9534b98d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37919 } }
I20260809 22:51:01.163295 24501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:01.163450 24501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:01.163678 24501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:01.166492 24501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fe7043c2184abd9b6ee25d9534b98d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37919 } }
I20260809 22:51:01.167066 24501 leader_election.cc:302] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97fe7043c2184abd9b6ee25d9534b98d; no voters:
I20260809 22:51:01.167779 24501 leader_election.cc:288] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:01.167889 24508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:01.168823 24508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [term 1 LEADER]: Becoming Leader. State: Replica: 97fe7043c2184abd9b6ee25d9534b98d, State: Running, Role: LEADER
I20260809 22:51:01.169404 24508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fe7043c2184abd9b6ee25d9534b98d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37919 } }
I20260809 22:51:01.170077 24501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:01.172170 24511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97fe7043c2184abd9b6ee25d9534b98d. Latest consensus state: current_term: 1 leader_uuid: "97fe7043c2184abd9b6ee25d9534b98d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fe7043c2184abd9b6ee25d9534b98d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37919 } } }
I20260809 22:51:01.172724 24509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97fe7043c2184abd9b6ee25d9534b98d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fe7043c2184abd9b6ee25d9534b98d" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37919 } } }
I20260809 22:51:01.173075 24511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:01.173194 24509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:01.174307 24524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:01.179337 24524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:01.190197 24524 catalog_manager.cc:1347] Generated new cluster ID: 276732d2712a4bb7ad780f84be8abe02
I20260809 22:51:01.190354 24524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:01.219761 24524 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:01.221787 24524 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:01.238775 24524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d: Generated new TSK 0
I20260809 22:51:01.240052 24524 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:51:01.319416 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37919
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:01.689659 24544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:01.689982 24544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:01.690125 24544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:01.697781 24544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:01.697927 24544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:01.698086 24544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:51:01.708333 24544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:01.709700 24544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:01.711061 24544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:01.724280 24549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:01.724314 24550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:01.724355 24553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:01.725265 24544 server_base.cc:1034] running on GCE node
I20260809 22:51:01.725849 24544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:01.726811 24544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:01.728024 24544 hybrid_clock.cc:648] HybridClock initialized: now 1786315861727960 us; error 56 us; skew 500 ppm
I20260809 22:51:01.728366 24544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:01 dist-test-slave-pww0 krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315861, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:01.731631 24544 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:51:01.733273 24544 webserver.cc:460] Webserver started at http://127.24.136.1:42389/ using document root <none> and password file <none>
I20260809 22:51:01.733976 24544 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:01.734100 24544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:01.734449 24544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:01.737533 24544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "682d81736f7b48b79b3c7e342dfd1ec3"
format_stamp: "Formatted at 2026-08-09 22:51:01 on dist-test-slave-pww0"
server_key: "949c607c75916de0a13316d10a8b7362"
server_key_iv: "ce4d96e16a5f50afea8b7b0e6004b4ad"
server_key_version: "encryptionkey@0"
I20260809 22:51:01.738412 24544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "682d81736f7b48b79b3c7e342dfd1ec3"
format_stamp: "Formatted at 2026-08-09 22:51:01 on dist-test-slave-pww0"
server_key: "949c607c75916de0a13316d10a8b7362"
server_key_iv: "ce4d96e16a5f50afea8b7b0e6004b4ad"
server_key_version: "encryptionkey@0"
I20260809 22:51:01.743952 24544 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:51:01.748250 24565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:01.749888 24544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:01.750082 24544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "682d81736f7b48b79b3c7e342dfd1ec3"
format_stamp: "Formatted at 2026-08-09 22:51:01 on dist-test-slave-pww0"
server_key: "949c607c75916de0a13316d10a8b7362"
server_key_iv: "ce4d96e16a5f50afea8b7b0e6004b4ad"
server_key_version: "encryptionkey@0"
I20260809 22:51:01.750316 24544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:01.762228 24544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:01.766958 24544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:01.767436 24544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:01.768601 24544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:01.770097 24544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:01.770176 24544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:01.770342 24544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:01.770411 24544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:01.801860 24544 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:33055
I20260809 22:51:01.801893 24765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:33055 every 8 connection(s)
I20260809 22:51:01.803211 24544 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:51:01.804301 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 24544
I20260809 22:51:01.804495 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:51:01.805014 25120 external_mini_cluster.cc:1428] Setting key beb64a565fbb47ca8b193cfb20a15948
Aug 09 22:51:01 dist-test-slave-pww0 krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315861, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:01.826454 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:01.806054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:47367 (local address 127.24.136.62:37919)
0809 22:51:01.806571 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:01.806577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:01.807812 (+ 1235us) server_negotiation.cc:407] Connection header received
0809 22:51:01.810527 (+ 2715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:01.810533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:01.810641 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:01.810767 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:01.812642 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:01.813623 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:01.815413 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:01.816024 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:01.816173 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:01.820274 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:01.820282 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:01.820286 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:01.820340 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:01.822992 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:01.823961 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:01.823966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:01.823968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:01.824110 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:01.824580 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:01.824584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:01.824586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:01.824864 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:01.825006 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:01.825821 (+ 815us) server_negotiation.cc:299] Negotiation successful
0809 22:51:01.826086 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":261,"threads_started":1}
I20260809 22:51:01.827503 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:01.806657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:37919 (local address 127.24.136.1:47367)
0809 22:51:01.807591 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:01.807640 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:01.810185 (+ 2545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:01.811075 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:01.811090 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:01.811750 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:01.812412 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:01.812432 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:01.813855 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:01.813860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:01.815206 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:01.815227 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:01.816303 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:01.816309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:01.816544 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:01.817256 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:01.817283 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:01.819961 (+ 2678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:01.823190 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:01.823201 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:01.823219 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:01.823810 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:01.824230 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:01.824236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:01.824239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:01.824448 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:01.825002 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:01.825011 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:01.825395 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 22:51:01.825663 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 22:51:01.826250 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":361,"threads_started":1}
I20260809 22:51:01.829387 24766 heartbeater.cc:347] Connected to a master server at 127.24.136.62:37919
I20260809 22:51:01.829756 24766 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:01.830564 24766 heartbeater.cc:511] Master 127.24.136.62:37919 requested a full tablet report, sending...
I20260809 22:51:01.833065 24440 ts_manager.cc:194] Registered new tserver with Master: 682d81736f7b48b79b3c7e342dfd1ec3 (127.24.136.1:33055)
I20260809 22:51:01.834853 24440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:47367
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:51:01.876962 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37919
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:02.247368 24774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:02.247746 24774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:02.247889 24774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:02.255604 24774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:02.255739 24774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:02.255903 24774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:51:02.266575 24774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:02.268113 24774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:02.269572 24774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:02.284519 24783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:02.284473 24782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:02.284550 24787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:02.285066 24774 server_base.cc:1034] running on GCE node
I20260809 22:51:02.285966 24774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:02.287076 24774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:02.288295 24774 hybrid_clock.cc:648] HybridClock initialized: now 1786315862288230 us; error 61 us; skew 500 ppm
I20260809 22:51:02.288650 24774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:02 dist-test-slave-pww0 krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315862, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:02.292091 24774 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:51:02.293730 24774 webserver.cc:460] Webserver started at http://127.24.136.2:44353/ using document root <none> and password file <none>
I20260809 22:51:02.294504 24774 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:02.294646 24774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:02.295003 24774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:02.298307 24774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "2794e4bdbdfc408d882bf6170b75cb40"
format_stamp: "Formatted at 2026-08-09 22:51:02 on dist-test-slave-pww0"
server_key: "d2053132e0cc00c474431fe3f7f10250"
server_key_iv: "61bf60c9d3cb9670b0c94ee99ffc43ba"
server_key_version: "encryptionkey@0"
I20260809 22:51:02.299199 24774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "2794e4bdbdfc408d882bf6170b75cb40"
format_stamp: "Formatted at 2026-08-09 22:51:02 on dist-test-slave-pww0"
server_key: "d2053132e0cc00c474431fe3f7f10250"
server_key_iv: "61bf60c9d3cb9670b0c94ee99ffc43ba"
server_key_version: "encryptionkey@0"
I20260809 22:51:02.305804 24774 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 22:51:02.309966 24798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:02.311764 24774 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 22:51:02.311972 24774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "2794e4bdbdfc408d882bf6170b75cb40"
format_stamp: "Formatted at 2026-08-09 22:51:02 on dist-test-slave-pww0"
server_key: "d2053132e0cc00c474431fe3f7f10250"
server_key_iv: "61bf60c9d3cb9670b0c94ee99ffc43ba"
server_key_version: "encryptionkey@0"
I20260809 22:51:02.312207 24774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:02.326471 24774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:02.331178 24774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:02.331625 24774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:02.333081 24774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:02.334981 24774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:02.335088 24774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:02.335212 24774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:02.335281 24774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:02.367991 24774 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:35065
I20260809 22:51:02.368141 24993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:35065 every 8 connection(s)
I20260809 22:51:02.369449 24774 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:51:02.370803 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 24774
I20260809 22:51:02.371016 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:51:02.371681 25120 external_mini_cluster.cc:1428] Setting key f82f1b18cae62aee5e6935c9dddb287a
Aug 09 22:51:02 dist-test-slave-pww0 krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315862, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:02.395709 24999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:02.373078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:40475 (local address 127.24.136.62:37919)
0809 22:51:02.373536 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:02.373540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:02.375009 (+ 1469us) server_negotiation.cc:407] Connection header received
0809 22:51:02.377699 (+ 2690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:02.377704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:02.377790 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:02.377893 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:02.380156 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.380944 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.383307 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.383834 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.383947 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:02.388629 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:02.388636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:02.388639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:02.388684 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:02.391842 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:02.392981 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:02.392985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:02.392988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:02.393139 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:02.393674 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:02.393677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:02.393679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:02.393977 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:02.394097 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:02.395148 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0809 22:51:02.395362 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":238,"threads_started":1}
I20260809 22:51:02.397073 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:02.373771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:37919 (local address 127.24.136.2:40475)
0809 22:51:02.374810 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:02.374870 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:02.377299 (+ 2429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:02.378210 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:02.378222 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:02.379007 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:02.379914 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:02.379942 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.381150 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.381155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:02.383101 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:02.383122 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.384110 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.384118 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:02.384441 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:02.385256 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:02.385288 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:02.388274 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:02.392071 (+ 3797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:02.392084 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:02.392107 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:02.392831 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:02.393276 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:02.393281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:02.393284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:02.393568 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:02.394125 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:02.394135 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:02.394653 (+ 518us) client_negotiation.cc:769] Sending connection context
0809 22:51:02.394986 (+ 333us) client_negotiation.cc:240] Negotiation successful
0809 22:51:02.395597 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":385,"threads_started":1}
I20260809 22:51:02.398775 24994 heartbeater.cc:347] Connected to a master server at 127.24.136.62:37919
I20260809 22:51:02.399168 24994 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:02.400077 24994 heartbeater.cc:511] Master 127.24.136.62:37919 requested a full tablet report, sending...
I20260809 22:51:02.401837 24440 ts_manager.cc:194] Registered new tserver with Master: 2794e4bdbdfc408d882bf6170b75cb40 (127.24.136.2:35065)
I20260809 22:51:02.402714 24440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:40475
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:51:02.447952 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37919
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:02.811545 25007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:02.811985 25007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:02.812150 25007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:02.820080 25007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:02.820204 25007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:02.820353 25007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:51:02.830680 25007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:02.832085 25007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:02.833433 25007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 22:51:02.838678 24766 heartbeater.cc:503] Master 127.24.136.62:37919 was elected leader, sending a full tablet report...
W20260809 22:51:02.847776 25021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:02.847856 25017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:02.847808 25018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:02.853432 25007 server_base.cc:1034] running on GCE node
I20260809 22:51:02.854240 25007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:02.855363 25007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:02.856624 25007 hybrid_clock.cc:648] HybridClock initialized: now 1786315862856547 us; error 76 us; skew 500 ppm
I20260809 22:51:02.856997 25007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:02 dist-test-slave-pww0 krb5kdc[24361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315862, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:02.860596 25007 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:51:02.862321 25007 webserver.cc:460] Webserver started at http://127.24.136.3:43571/ using document root <none> and password file <none>
I20260809 22:51:02.863099 25007 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:02.863227 25007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:02.863626 25007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:02.866794 25007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "95cbef569069434fa615702e060e7380"
format_stamp: "Formatted at 2026-08-09 22:51:02 on dist-test-slave-pww0"
server_key: "cd9eb0e0b9d6fccf786e75ad2fcef846"
server_key_iv: "918efbf74908911457b2ef590e516bd5"
server_key_version: "encryptionkey@0"
I20260809 22:51:02.867673 25007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "95cbef569069434fa615702e060e7380"
format_stamp: "Formatted at 2026-08-09 22:51:02 on dist-test-slave-pww0"
server_key: "cd9eb0e0b9d6fccf786e75ad2fcef846"
server_key_iv: "918efbf74908911457b2ef590e516bd5"
server_key_version: "encryptionkey@0"
I20260809 22:51:02.873561 25007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:51:02.877712 25032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:02.879297 25007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:51:02.879491 25007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "95cbef569069434fa615702e060e7380"
format_stamp: "Formatted at 2026-08-09 22:51:02 on dist-test-slave-pww0"
server_key: "cd9eb0e0b9d6fccf786e75ad2fcef846"
server_key_iv: "918efbf74908911457b2ef590e516bd5"
server_key_version: "encryptionkey@0"
I20260809 22:51:02.879712 25007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:02.907845 25007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:02.912786 25007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:02.913106 25007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:02.914178 25007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:02.915737 25007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:02.915817 25007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:02.915923 25007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:02.915982 25007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:02.948139 25007 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:36587
I20260809 22:51:02.948211 25232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:36587 every 8 connection(s)
I20260809 22:51:02.949489 25007 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:51:02.952021 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 25007
I20260809 22:51:02.952232 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:51:02.952805 25120 external_mini_cluster.cc:1428] Setting key e7b49aca93fcd6e552445f8705e4d26c
Aug 09 22:51:02 dist-test-slave-pww0 krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315862, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:02.971487 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:02.952257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:51789 (local address 127.24.136.62:37919)
0809 22:51:02.952815 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:02.952821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:02.953820 (+ 999us) server_negotiation.cc:407] Connection header received
0809 22:51:02.956282 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:02.956288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:02.956382 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:02.956474 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:02.958042 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.958841 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.960501 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.960933 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.961042 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:02.964789 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:02.964795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:02.964798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:02.964836 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:02.967666 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:02.968730 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:02.968734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:02.968737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:02.968879 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:02.969531 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:02.969536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:02.969538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:02.969860 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:02.970057 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:02.970911 (+ 854us) server_negotiation.cc:299] Negotiation successful
0809 22:51:02.971159 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":243,"threads_started":1}
I20260809 22:51:02.972467 25236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:02.952792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:37919 (local address 127.24.136.3:51789)
0809 22:51:02.953618 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:02.953669 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:02.955954 (+ 2285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:02.956714 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:02.956723 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:02.957270 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:02.957887 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:02.957907 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.958966 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.958970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:02.960327 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:02.960340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.961130 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.961135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:02.961370 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:02.962008 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:02.962032 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:02.964510 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:02.967852 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:02.967864 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:02.967886 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:02.968555 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:02.969023 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:02.969029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:02.969033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:02.969371 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:02.970053 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:02.970061 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:02.970491 (+ 430us) client_negotiation.cc:769] Sending connection context
0809 22:51:02.970758 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 22:51:02.971284 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":311,"threads_started":1}
I20260809 22:51:02.974076 25233 heartbeater.cc:347] Connected to a master server at 127.24.136.62:37919
I20260809 22:51:02.974426 25233 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:02.975251 25233 heartbeater.cc:511] Master 127.24.136.62:37919 requested a full tablet report, sending...
I20260809 22:51:02.976991 24440 ts_manager.cc:194] Registered new tserver with Master: 95cbef569069434fa615702e060e7380 (127.24.136.3:36587)
I20260809 22:51:02.977876 24440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:51789
I20260809 22:51:02.982252 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:51:03.000515 25235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:02.987005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.24.136.62:37919)
0809 22:51:02.987245 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:02.987261 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:02.987591 (+ 330us) server_negotiation.cc:407] Connection header received
0809 22:51:02.987756 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:02.987761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:02.987836 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:02.987926 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:02.989468 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.990218 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.991736 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:02.992223 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:02.992336 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:02.994998 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:02.995003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:02.995006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:02.995042 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:02.997699 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:02.998474 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:02.998477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:02.998480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:02.998601 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:02.998954 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:02.998958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:02.998960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:02.999284 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:02.999550 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.000000 (+ 450us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.000180 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:51:03.004987 24440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:51:03.012284 24440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:51:03.037469 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.027366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33055 (local address 127.0.0.1:43608)
0809 22:51:03.028130 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:03.028147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:03.028258 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:03.030128 (+ 1870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:03.030133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:03.030160 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:03.030562 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.030574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.032501 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.032510 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.034640 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.034658 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.036455 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.036462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.036666 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.036714 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 22:51:03.036805 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:51:03.036977 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":77,"thread_start_us":528,"threads_started":1}
I20260809 22:51:03.038414 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.026569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44030 (local address 127.24.136.2:35065)
0809 22:51:03.028740 (+ 2171us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.028748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.028778 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:51:03.028873 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.028880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.029139 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.030166 (+ 1027us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:03.030441 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.031971 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.035377 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.037332 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.037464 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.037679 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.037945 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.038128 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2008,"thread_start_us":1901,"threads_started":1}
I20260809 22:51:03.038476 25256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.025873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:35065 (local address 127.0.0.1:44030)
0809 22:51:03.027258 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:03.027276 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:03.027386 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:03.029398 (+ 2012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:03.029403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:03.029459 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:03.029958 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.029972 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.032148 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.032153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.035145 (+ 2992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.035164 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.037568 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.037575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.037750 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.037778 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:51:03.037868 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 22:51:03.038008 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"mutex_wait_us":183,"spinlock_wait_cycles":5760,"thread_start_us":1129,"threads_started":1}
I20260809 22:51:03.038852 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.028046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.24.136.1:33055)
0809 22:51:03.029515 (+ 1469us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.029524 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.029552 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:51:03.029649 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.029657 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.029900 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.030124 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:03.030746 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.032340 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.034850 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.036213 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.038046 (+ 1833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.038290 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.038364 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.038536 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1285,"thread_start_us":1132,"threads_started":1}
I20260809 22:51:03.038863 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.025741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:36587 (local address 127.0.0.1:52022)
0809 22:51:03.026568 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:03.026622 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:03.026823 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:03.029382 (+ 2559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:03.029390 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:03.029447 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:03.029871 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.029882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.032027 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.032032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.035146 (+ 3114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.035164 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.037438 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.037445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.037912 (+ 467us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.037944 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:51:03.038031 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 22:51:03.038179 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":237,"spinlock_wait_cycles":4480,"thread_start_us":411,"threads_started":1}
I20260809 22:51:03.039590 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.025963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52022 (local address 127.24.136.3:36587)
0809 22:51:03.027497 (+ 1534us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.027508 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.027546 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:51:03.028758 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.028770 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.029091 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.030050 (+ 959us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:03.030469 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.031864 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.035377 (+ 3513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.036826 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.036972 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.037229 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.039147 (+ 1918us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.039301 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1324,"thread_start_us":1172,"threads_started":1}
I20260809 22:51:03.042501 24888 tablet_service.cc:1467] Processing CreateTablet for tablet f431a1c2d0d448b2a72abf09ddc4db3b (DEFAULT_TABLE table=test-table [id=a287a274a4f042d8a8855dde7d413d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:03.043290 24645 tablet_service.cc:1467] Processing CreateTablet for tablet f431a1c2d0d448b2a72abf09ddc4db3b (DEFAULT_TABLE table=test-table [id=a287a274a4f042d8a8855dde7d413d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:03.043290 25114 tablet_service.cc:1467] Processing CreateTablet for tablet f431a1c2d0d448b2a72abf09ddc4db3b (DEFAULT_TABLE table=test-table [id=a287a274a4f042d8a8855dde7d413d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:03.044003 24888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f431a1c2d0d448b2a72abf09ddc4db3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:03.044862 25114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f431a1c2d0d448b2a72abf09ddc4db3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:03.044864 24645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f431a1c2d0d448b2a72abf09ddc4db3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:03.057063 25265 tablet_bootstrap.cc:492] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: Bootstrap starting.
I20260809 22:51:03.061209 25265 tablet_bootstrap.cc:654] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:03.062422 25265 log.cc:824] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:03.065234 25267 tablet_bootstrap.cc:492] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: Bootstrap starting.
I20260809 22:51:03.065644 25265 tablet_bootstrap.cc:492] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: No bootstrap required, opened a new log
I20260809 22:51:03.065922 25265 ts_tablet_manager.cc:1397] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:51:03.067957 25268 tablet_bootstrap.cc:492] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: Bootstrap starting.
I20260809 22:51:03.068569 25267 tablet_bootstrap.cc:654] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:03.069988 25267 log.cc:824] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:03.071797 25268 tablet_bootstrap.cc:654] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:03.073184 25268 log.cc:824] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:03.073835 25267 tablet_bootstrap.cc:492] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: No bootstrap required, opened a new log
I20260809 22:51:03.074134 25267 ts_tablet_manager.cc:1397] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:51:03.074566 25265 raft_consensus.cc:348] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
I20260809 22:51:03.075091 25265 raft_consensus.cc:374] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:03.075193 25265 raft_consensus.cc:729] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2794e4bdbdfc408d882bf6170b75cb40, State: Initialized, Role: FOLLOWER
I20260809 22:51:03.075925 25265 consensus_queue.cc:260] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
I20260809 22:51:03.076838 25268 tablet_bootstrap.cc:492] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: No bootstrap required, opened a new log
I20260809 22:51:03.077276 24994 heartbeater.cc:503] Master 127.24.136.62:37919 was elected leader, sending a full tablet report...
I20260809 22:51:03.077770 25265 ts_tablet_manager.cc:1428] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: Time spent starting tablet: real 0.012s user 0.002s sys 0.010s
I20260809 22:51:03.078037 25268 ts_tablet_manager.cc:1397] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260809 22:51:03.084807 25267 raft_consensus.cc:348] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
I20260809 22:51:03.085351 25267 raft_consensus.cc:374] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:03.085464 25267 raft_consensus.cc:729] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 682d81736f7b48b79b3c7e342dfd1ec3, State: Initialized, Role: FOLLOWER
I20260809 22:51:03.086161 25267 consensus_queue.cc:260] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
I20260809 22:51:03.088120 25267 ts_tablet_manager.cc:1428] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260809 22:51:03.088677 25268 raft_consensus.cc:348] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
I20260809 22:51:03.089110 25268 raft_consensus.cc:374] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:03.089223 25268 raft_consensus.cc:729] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95cbef569069434fa615702e060e7380, State: Initialized, Role: FOLLOWER
I20260809 22:51:03.089920 25268 consensus_queue.cc:260] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
I20260809 22:51:03.091127 25233 heartbeater.cc:503] Master 127.24.136.62:37919 was elected leader, sending a full tablet report...
I20260809 22:51:03.091840 25268 ts_tablet_manager.cc:1428] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 22:51:03.114467 25279 raft_consensus.cc:482] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:51:03.114789 25279 raft_consensus.cc:504] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
I20260809 22:51:03.116561 25279 leader_election.cc:288] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95cbef569069434fa615702e060e7380 (127.24.136.3:36587), 2794e4bdbdfc408d882bf6170b75cb40 (127.24.136.2:35065)
I20260809 22:51:03.120504 25275 raft_consensus.cc:482] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:51:03.120862 25275 raft_consensus.cc:504] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
W20260809 22:51:03.122802 24995 tablet.cc:2369] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:51:03.123000 25275 leader_election.cc:288] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95cbef569069434fa615702e060e7380 (127.24.136.3:36587), 682d81736f7b48b79b3c7e342dfd1ec3 (127.24.136.1:33055)
I20260809 22:51:03.134030 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.117026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:38307 (local address 127.24.136.2:35065)
0809 22:51:03.117262 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.117269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.117843 (+ 574us) server_negotiation.cc:407] Connection header received
0809 22:51:03.118064 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.118070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.118154 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.118256 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:03.118998 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.120170 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.122835 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.124184 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.124297 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.124372 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.129823 (+ 5451us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.132875 (+ 3052us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:51:03.136022 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.116842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:35065 (local address 127.24.136.1:38307)
0809 22:51:03.117718 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:03.117742 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:03.117915 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:03.118282 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:03.118287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:03.118323 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:03.118878 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.118888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.120364 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.120370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.122606 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.122625 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.129252 (+ 6627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.129260 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.129496 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.129544 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 22:51:03.135284 (+ 5740us) client_negotiation.cc:240] Negotiation successful
0809 22:51:03.135507 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":25,"spinlock_wait_cycles":1280,"thread_start_us":721,"threads_started":1}
I20260809 22:51:03.137692 24923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a1c2d0d448b2a72abf09ddc4db3b" candidate_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2794e4bdbdfc408d882bf6170b75cb40" is_pre_election: true
I20260809 22:51:03.138230 24923 raft_consensus.cc:2454] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 682d81736f7b48b79b3c7e342dfd1ec3 in term 0.
I20260809 22:51:03.139089 24568 leader_election.cc:302] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2794e4bdbdfc408d882bf6170b75cb40, 682d81736f7b48b79b3c7e342dfd1ec3; no voters:
I20260809 22:51:03.139691 25279 raft_consensus.cc:2781] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:51:03.139870 25279 raft_consensus.cc:482] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:51:03.139977 25279 raft_consensus.cc:3037] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:03.140218 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.117198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:51171 (local address 127.24.136.3:36587)
0809 22:51:03.117404 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.117409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.117942 (+ 533us) server_negotiation.cc:407] Connection header received
0809 22:51:03.118065 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.118070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.118153 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.118264 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:03.118980 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.120178 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.137566 (+ 17388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.138798 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.138974 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.139059 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.139811 (+ 752us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.139962 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:51:03.141413 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.116871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:36587 (local address 127.24.136.1:51171)
0809 22:51:03.117830 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:03.117849 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:03.117947 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:03.118282 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:03.118287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:03.118329 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:03.118807 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.118825 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.121264 (+ 2439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.121271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.137091 (+ 15820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.137124 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.139367 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.139374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.139589 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.139624 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:51:03.140795 (+ 1171us) client_negotiation.cc:240] Negotiation successful
0809 22:51:03.140961 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":10761,"spinlock_wait_cycles":3840,"thread_start_us":457,"threads_started":1}
I20260809 22:51:03.143510 25279 raft_consensus.cc:504] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
I20260809 22:51:03.145402 24923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a1c2d0d448b2a72abf09ddc4db3b" candidate_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2794e4bdbdfc408d882bf6170b75cb40"
I20260809 22:51:03.145687 24923 raft_consensus.cc:3037] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:03.146883 25150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a1c2d0d448b2a72abf09ddc4db3b" candidate_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95cbef569069434fa615702e060e7380" is_pre_election: true
I20260809 22:51:03.147607 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.124008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:41967 (local address 127.24.136.3:36587)
0809 22:51:03.140427 (+ 16419us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.140432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.140450 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:51:03.140506 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.140510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.140584 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.140676 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:03.141269 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.142375 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.144945 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.146225 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.146368 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.146444 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.147111 (+ 667us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.147264 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16322,"thread_start_us":21788,"threads_started":1}
I20260809 22:51:03.147536 25150 raft_consensus.cc:2454] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 682d81736f7b48b79b3c7e342dfd1ec3 in term 0.
I20260809 22:51:03.148272 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.124733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:35203 (local address 127.24.136.1:33055)
0809 22:51:03.124933 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.124939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.137766 (+ 12827us) server_negotiation.cc:407] Connection header received
0809 22:51:03.137948 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.137953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.138038 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.138092 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:03.138912 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.140074 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.145810 (+ 5736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.147032 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.147162 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.147231 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.147894 (+ 663us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.148030 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 22:51:03.148381 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.123693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:36587 (local address 127.24.136.2:41967)
0809 22:51:03.136891 (+ 13198us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:03.136914 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:03.137050 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:03.140692 (+ 3642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:03.140697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:03.140726 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:03.141110 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.141121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.142574 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.142578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.144743 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.144770 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.146677 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.146684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.146886 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.146928 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:51:03.147787 (+ 859us) client_negotiation.cc:240] Negotiation successful
0809 22:51:03.147942 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13047,"mutex_wait_us":32,"thread_start_us":13632,"threads_started":1}
I20260809 22:51:03.148358 25150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a1c2d0d448b2a72abf09ddc4db3b" candidate_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95cbef569069434fa615702e060e7380"
I20260809 22:51:03.148546 25150 raft_consensus.cc:3037] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:03.149045 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.124243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:33055 (local address 127.24.136.2:35203)
0809 22:51:03.137646 (+ 13403us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:03.137660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:03.137793 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:03.138269 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:03.138274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:03.138299 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:03.138711 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.138722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.140232 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.140237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.145599 (+ 5362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:03.145615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.147493 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.147501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:03.147701 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.147738 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:51:03.148547 (+ 809us) client_negotiation.cc:240] Negotiation successful
0809 22:51:03.148691 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13297,"thread_start_us":12292,"threads_started":1}
I20260809 22:51:03.149267 25279 leader_election.cc:288] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [CANDIDATE]: Term 1 election: Requested vote from peers 95cbef569069434fa615702e060e7380 (127.24.136.3:36587), 2794e4bdbdfc408d882bf6170b75cb40 (127.24.136.2:35065)
I20260809 22:51:03.150055 25149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a1c2d0d448b2a72abf09ddc4db3b" candidate_uuid: "2794e4bdbdfc408d882bf6170b75cb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95cbef569069434fa615702e060e7380" is_pre_election: true
I20260809 22:51:03.150058 24691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f431a1c2d0d448b2a72abf09ddc4db3b" candidate_uuid: "2794e4bdbdfc408d882bf6170b75cb40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" is_pre_election: true
I20260809 22:51:03.150563 24691 raft_consensus.cc:2379] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2794e4bdbdfc408d882bf6170b75cb40 in current term 1: Already voted for candidate 682d81736f7b48b79b3c7e342dfd1ec3 in this term.
I20260809 22:51:03.150998 24923 raft_consensus.cc:2454] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 682d81736f7b48b79b3c7e342dfd1ec3 in term 1.
I20260809 22:51:03.151405 24568 leader_election.cc:302] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2794e4bdbdfc408d882bf6170b75cb40, 682d81736f7b48b79b3c7e342dfd1ec3; no voters:
I20260809 22:51:03.151623 24800 leader_election.cc:302] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2794e4bdbdfc408d882bf6170b75cb40; no voters: 682d81736f7b48b79b3c7e342dfd1ec3, 95cbef569069434fa615702e060e7380
I20260809 22:51:03.151742 25279 raft_consensus.cc:2781] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:03.152001 25150 raft_consensus.cc:2454] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 682d81736f7b48b79b3c7e342dfd1ec3 in term 1.
I20260809 22:51:03.152094 25275 raft_consensus.cc:2726] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 22:51:03.152534 25279 raft_consensus.cc:686] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 1 LEADER]: Becoming Leader. State: Replica: 682d81736f7b48b79b3c7e342dfd1ec3, State: Running, Role: LEADER
I20260809 22:51:03.153035 25279 consensus_queue.cc:237] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } }
I20260809 22:51:03.158082 24440 catalog_manager.cc:5495] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 682d81736f7b48b79b3c7e342dfd1ec3 (127.24.136.1). New cstate: current_term: 1 leader_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "682d81736f7b48b79b3c7e342dfd1ec3" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 33055 } health_report { overall_health: HEALTHY } } }
W20260809 22:51:03.201218 25234 tablet.cc:2369] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:51:03.202986 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.196994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43612 (local address 127.24.136.1:33055)
0809 22:51:03.197181 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.197186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.197206 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:51:03.197357 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.197362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.197442 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.197579 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:03.198030 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.198831 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.200440 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.200877 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.200982 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.201702 (+ 720us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:03.202209 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:03.202356 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.202603 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.202685 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:51:03.212338 25150 raft_consensus.cc:1261] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [term 1 FOLLOWER]: Refusing update from remote peer 682d81736f7b48b79b3c7e342dfd1ec3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:03.212399 24923 raft_consensus.cc:1261] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 1 FOLLOWER]: Refusing update from remote peer 682d81736f7b48b79b3c7e342dfd1ec3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:03.213420 25279 consensus_queue.cc:1035] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95cbef569069434fa615702e060e7380" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 36587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:51:03.213800 25292 consensus_queue.cc:1035] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2794e4bdbdfc408d882bf6170b75cb40" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 35065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:51:03.243829 24440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:51:03.244251 24440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:51:03.248453 24440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97fe7043c2184abd9b6ee25d9534b98d: Sending DeleteTablet for 3 replicas of tablet f431a1c2d0d448b2a72abf09ddc4db3b
I20260809 22:51:03.249795 25114 tablet_service.cc:1514] Processing DeleteTablet for tablet f431a1c2d0d448b2a72abf09ddc4db3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:03 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:52022
I20260809 22:51:03.249802 24888 tablet_service.cc:1514] Processing DeleteTablet for tablet f431a1c2d0d448b2a72abf09ddc4db3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:03 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:44030
I20260809 22:51:03.249927 24645 tablet_service.cc:1514] Processing DeleteTablet for tablet f431a1c2d0d448b2a72abf09ddc4db3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:03 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:43608
I20260809 22:51:03.252182 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 24544
I20260809 22:51:03.262399 25310 tablet_replica.cc:330] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: stopping tablet replica
I20260809 22:51:03.263115 25310 raft_consensus.cc:2229] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:51:03.263855 25310 raft_consensus.cc:2258] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:03.264791 25311 tablet_replica.cc:330] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: stopping tablet replica
I20260809 22:51:03.267114 25310 ts_tablet_manager.cc:1905] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:03.270344 25311 raft_consensus.cc:2229] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:51:03.271111 25311 raft_consensus.cc:2258] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:03.277649 25313 tablet_replica.cc:330] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: stopping tablet replica
I20260809 22:51:03.277716 25311 ts_tablet_manager.cc:1905] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:03.278182 25313 raft_consensus.cc:2229] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:51:03.279032 25313 raft_consensus.cc:2258] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:03.281509 25310 ts_tablet_manager.cc:1918] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:51:03.281729 25310 log.cc:1196] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/f431a1c2d0d448b2a72abf09ddc4db3b
I20260809 22:51:03.281745 25313 ts_tablet_manager.cc:1905] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:03.282310 25310 ts_tablet_manager.cc:1939] T f431a1c2d0d448b2a72abf09ddc4db3b P 2794e4bdbdfc408d882bf6170b75cb40: Deleting consensus metadata
Aug 09 22:51:03 dist-test-slave-pww0 krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:51:03.284395 24417 catalog_manager.cc:4841] TS 2794e4bdbdfc408d882bf6170b75cb40 (127.24.136.2:35065): tablet f431a1c2d0d448b2a72abf09ddc4db3b (table test-table [id=a287a274a4f042d8a8855dde7d413d07]) successfully deleted
I20260809 22:51:03.289741 25311 ts_tablet_manager.cc:1918] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:51:03.289951 25311 log.cc:1196] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/f431a1c2d0d448b2a72abf09ddc4db3b
I20260809 22:51:03.290582 25311 ts_tablet_manager.cc:1939] T f431a1c2d0d448b2a72abf09ddc4db3b P 95cbef569069434fa615702e060e7380: Deleting consensus metadata
I20260809 22:51:03.290773 25313 ts_tablet_manager.cc:1918] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:51:03.291034 25313 log.cc:1196] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/f431a1c2d0d448b2a72abf09ddc4db3b
I20260809 22:51:03.291738 25313 ts_tablet_manager.cc:1939] T f431a1c2d0d448b2a72abf09ddc4db3b P 682d81736f7b48b79b3c7e342dfd1ec3: Deleting consensus metadata
I20260809 22:51:03.292729 24420 catalog_manager.cc:4841] TS 95cbef569069434fa615702e060e7380 (127.24.136.3:36587): tablet f431a1c2d0d448b2a72abf09ddc4db3b (table test-table [id=a287a274a4f042d8a8855dde7d413d07]) successfully deleted
I20260809 22:51:03.293543 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.253723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.24.136.1:33055)
0809 22:51:03.253917 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.253924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.268763 (+ 14839us) server_negotiation.cc:407] Connection header received
0809 22:51:03.269291 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.269298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.269422 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.269499 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:03.271691 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.272772 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.274788 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.275528 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.275694 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.284272 (+ 8578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:03.284292 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:03.284315 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:03.284378 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:03.288786 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:03.289898 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:03.289910 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:03.289913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:03.290141 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:03.290744 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:03.290749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:03.290757 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:03.291333 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:03.291701 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.293051 (+ 1350us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.293279 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:51:03.293781 24417 catalog_manager.cc:4841] TS 682d81736f7b48b79b3c7e342dfd1ec3 (127.24.136.1:33055): tablet f431a1c2d0d448b2a72abf09ddc4db3b (table test-table [id=a287a274a4f042d8a8855dde7d413d07]) successfully deleted
I20260809 22:51:03.294234 24760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:03.396824 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 24544
I20260809 22:51:03.427012 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 24774
Aug 09 22:51:03 dist-test-slave-pww0 krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:51:03.443910 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.427897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44038 (local address 127.24.136.2:35065)
0809 22:51:03.428093 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.428099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.428139 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:51:03.428261 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.428265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.428348 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.428449 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:03.429741 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.430519 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.431883 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.432316 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.432431 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.437056 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:03.437074 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:03.437092 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:03.437146 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:03.440727 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:03.441600 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:03.441627 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:03.441629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:03.441785 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:03.442211 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:03.442215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:03.442217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:03.442642 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:03.442938 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.443286 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.443545 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:51:03.444664 24988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:03.535846 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 24774
I20260809 22:51:03.558418 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 25007
Aug 09 22:51:03 dist-test-slave-pww0 krb5kdc[24361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:51:03.574086 25290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:03.559242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52028 (local address 127.24.136.3:36587)
0809 22:51:03.559510 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:03.559516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:03.559542 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:51:03.559621 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:03.559625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:03.559693 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:03.559802 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:03.561152 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.561896 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.563181 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:03.563651 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:03.563758 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:03.568138 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:03.568157 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:03.568174 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:03.568216 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:03.571153 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:03.572040 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:03.572049 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:03.572051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:03.572208 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:03.572636 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:03.572639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:03.572641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:03.573037 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:03.573285 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:03.573531 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 22:51:03.573720 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 22:51:03.574909 25225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:03.667301 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 25007
I20260809 22:51:03.690045 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 24381
I20260809 22:51:03.691212 24493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:03.802947 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 24381
2026-08-09T22:51:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4804 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T22:51:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:03Z Disabled control of system clock
I20260809 22:51:03.907233 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:42059
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:45497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:42059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:51:04.268333 25336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:04.268675 25336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:04.268790 25336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:04.276567 25336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:04.276706 25336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:04.276751 25336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:04.276834 25336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:04.276898 25336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:04.286753 25336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:42059
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:42059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:04.288204 25336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:04.290159 25336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:04.302138 25344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:04.302312 25348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:04.302325 25343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:04.303237 25336 server_base.cc:1034] running on GCE node
I20260809 22:51:04.304172 25336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:04.305692 25336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:04.306943 25336 hybrid_clock.cc:648] HybridClock initialized: now 1786315864306873 us; error 61 us; skew 500 ppm
I20260809 22:51:04.307354 25336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:51:04.309952 25336 webserver.cc:460] Webserver started at http://127.24.136.62:46125/ using document root <none> and password file <none>
I20260809 22:51:04.310747 25336 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:04.310863 25336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:04.311210 25336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:04.314574 25336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "c9f677f4c5834504a5c55e3350b985b4"
format_stamp: "Formatted at 2026-08-09 22:51:04 on dist-test-slave-pww0"
server_key: "0b4874c7c85fddeefd96d7e881626e4e"
server_key_iv: "0ad34b1f3bb5dc27a5725298466de992"
server_key_version: "encryptionkey@0"
I20260809 22:51:04.315560 25336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "c9f677f4c5834504a5c55e3350b985b4"
format_stamp: "Formatted at 2026-08-09 22:51:04 on dist-test-slave-pww0"
server_key: "0b4874c7c85fddeefd96d7e881626e4e"
server_key_iv: "0ad34b1f3bb5dc27a5725298466de992"
server_key_version: "encryptionkey@0"
I20260809 22:51:04.321225 25336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:51:04.325238 25361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:04.326785 25336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:51:04.326972 25336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "c9f677f4c5834504a5c55e3350b985b4"
format_stamp: "Formatted at 2026-08-09 22:51:04 on dist-test-slave-pww0"
server_key: "0b4874c7c85fddeefd96d7e881626e4e"
server_key_iv: "0ad34b1f3bb5dc27a5725298466de992"
server_key_version: "encryptionkey@0"
I20260809 22:51:04.327208 25336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:04.341658 25336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:04.342595 25336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:04.342903 25336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:04.361591 25336 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:42059
I20260809 22:51:04.361613 25464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:42059 every 8 connection(s)
I20260809 22:51:04.363070 25336 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:04.367182 25465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:04.369612 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 25336
I20260809 22:51:04.369824 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:04.370389 25120 external_mini_cluster.cc:1428] Setting key 21625eede275f7c4d7bcfdc2ab484464
I20260809 22:51:04.380318 25465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4: Bootstrap starting.
I20260809 22:51:04.383708 25465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:04.384910 25472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:04.372749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.24.136.62:42059)
0809 22:51:04.373899 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:04.373914 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:04.373966 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:51:04.375772 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:04.375799 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:04.376314 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:04.376714 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:04.377362 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:04.378768 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:04.380295 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:04.381058 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:04.381255 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:04.382451 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:04.382459 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:04.382474 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:04.383026 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:04.383130 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:04.383154 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:04.383275 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:04.383539 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 22:51:04.383887 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":375,"threads_started":1}
I20260809 22:51:04.384903 25465 log.cc:824] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:04.388170 25465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4: No bootstrap required, opened a new log
I20260809 22:51:04.396245 25465 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f677f4c5834504a5c55e3350b985b4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42059 } }
I20260809 22:51:04.396682 25465 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:04.396811 25465 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f677f4c5834504a5c55e3350b985b4, State: Initialized, Role: FOLLOWER
I20260809 22:51:04.397442 25465 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f677f4c5834504a5c55e3350b985b4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42059 } }
I20260809 22:51:04.397678 25465 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:04.397819 25465 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:04.397976 25465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:04.400554 25465 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f677f4c5834504a5c55e3350b985b4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42059 } }
I20260809 22:51:04.401149 25465 leader_election.cc:302] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9f677f4c5834504a5c55e3350b985b4; no voters:
I20260809 22:51:04.401863 25465 leader_election.cc:288] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:04.402025 25476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:04.402755 25476 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [term 1 LEADER]: Becoming Leader. State: Replica: c9f677f4c5834504a5c55e3350b985b4, State: Running, Role: LEADER
I20260809 22:51:04.403297 25476 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f677f4c5834504a5c55e3350b985b4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42059 } }
I20260809 22:51:04.404111 25465 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:04.406659 25478 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9f677f4c5834504a5c55e3350b985b4. Latest consensus state: current_term: 1 leader_uuid: "c9f677f4c5834504a5c55e3350b985b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f677f4c5834504a5c55e3350b985b4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42059 } } }
I20260809 22:51:04.406332 25477 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9f677f4c5834504a5c55e3350b985b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f677f4c5834504a5c55e3350b985b4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 42059 } } }
I20260809 22:51:04.407256 25477 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:04.407289 25478 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:04.408154 25493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:04.413928 25493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:04.428084 25493 catalog_manager.cc:1347] Generated new cluster ID: 4bd8541422b340d7bd34af7350ada949
I20260809 22:51:04.428234 25493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:04.442570 25493 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:04.444201 25493 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:04.450999 25493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4: Generated new TSK 0
I20260809 22:51:04.452247 25493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:04.463990 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42059
--builtin_ntp_servers=127.24.136.20:45497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:51:04.826921 25510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:04.827217 25510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:04.827306 25510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:04.834818 25510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:04.834933 25510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:04.835042 25510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:51:04.844434 25510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:04.845716 25510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:04.847733 25510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:04.861243 25522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:04.861208 25525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:04.861268 25518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:04.862475 25510 server_base.cc:1034] running on GCE node
I20260809 22:51:04.863150 25510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:04.864283 25510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:04.865525 25510 hybrid_clock.cc:648] HybridClock initialized: now 1786315864865466 us; error 53 us; skew 500 ppm
I20260809 22:51:04.865890 25510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:51:04.868381 25510 webserver.cc:460] Webserver started at http://127.24.136.1:42559/ using document root <none> and password file <none>
I20260809 22:51:04.869115 25510 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:04.869230 25510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:04.869555 25510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:04.872658 25510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "f917889777ac4108a1fea1a43050c06c"
format_stamp: "Formatted at 2026-08-09 22:51:04 on dist-test-slave-pww0"
server_key: "140b22674b0afef954b9938be8809871"
server_key_iv: "3cb91a8c43f9d5852ffb3175b676bc81"
server_key_version: "encryptionkey@0"
I20260809 22:51:04.873476 25510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "f917889777ac4108a1fea1a43050c06c"
format_stamp: "Formatted at 2026-08-09 22:51:04 on dist-test-slave-pww0"
server_key: "140b22674b0afef954b9938be8809871"
server_key_iv: "3cb91a8c43f9d5852ffb3175b676bc81"
server_key_version: "encryptionkey@0"
I20260809 22:51:04.878902 25510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:51:04.882655 25533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:04.884107 25510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:51:04.884275 25510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "f917889777ac4108a1fea1a43050c06c"
format_stamp: "Formatted at 2026-08-09 22:51:04 on dist-test-slave-pww0"
server_key: "140b22674b0afef954b9938be8809871"
server_key_iv: "3cb91a8c43f9d5852ffb3175b676bc81"
server_key_version: "encryptionkey@0"
I20260809 22:51:04.884488 25510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:04.905556 25510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:04.906490 25510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:04.906807 25510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:04.907929 25510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:04.909534 25510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:04.909652 25510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:04.909768 25510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:04.909842 25510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:04.941736 25510 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:45673
I20260809 22:51:04.941792 25734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:45673 every 8 connection(s)
I20260809 22:51:04.943245 25510 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:51:04.946250 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 25510
I20260809 22:51:04.946440 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:51:04.946988 25120 external_mini_cluster.cc:1428] Setting key 3e21084d6120d4d37e93b9a1c2aab25b
I20260809 22:51:04.950160 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42059
--builtin_ntp_servers=127.24.136.20:45497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:51:04.957696 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:04.946093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:39009 (local address 127.24.136.62:42059)
0809 22:51:04.946558 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:04.946563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:04.948213 (+ 1650us) server_negotiation.cc:407] Connection header received
0809 22:51:04.950128 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:04.950133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:04.950228 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:04.950370 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:04.951923 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:04.952919 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:04.954635 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:04.955200 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:04.955334 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:04.956827 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:04.956834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:04.956838 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:04.956864 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:04.956915 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:04.956928 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:04.957018 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:04.957229 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 22:51:04.957340 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":259,"threads_started":1}
I20260809 22:51:04.958710 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:04.946836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42059 (local address 127.24.136.1:39009)
0809 22:51:04.948005 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:04.948068 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:04.949829 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:04.950812 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:04.950821 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:04.950926 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:04.951723 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:04.951746 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:04.953107 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:04.953111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:04.954465 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:04.954482 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:04.955441 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:04.955446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:04.955717 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:04.956449 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:51:04.956451 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:04.956497 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:51:04.956597 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:04.957018 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:04.957025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:04.957031 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:51:04.957178 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 22:51:04.957542 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":345,"threads_started":1}
I20260809 22:51:04.960393 25735 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42059
I20260809 22:51:04.960723 25735 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:04.961537 25735 heartbeater.cc:511] Master 127.24.136.62:42059 requested a full tablet report, sending...
I20260809 22:51:04.963997 25394 ts_manager.cc:194] Registered new tserver with Master: f917889777ac4108a1fea1a43050c06c (127.24.136.1:45673)
I20260809 22:51:04.965660 25394 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.1:39009
W20260809 22:51:05.313614 25745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:05.313946 25745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:05.314083 25745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:05.321590 25745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:05.321735 25745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:05.321893 25745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:51:05.331734 25745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:05.333267 25745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:05.335278 25745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:05.350383 25756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:05.350343 25759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:05.350404 25755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:05.351168 25745 server_base.cc:1034] running on GCE node
I20260809 22:51:05.351886 25745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:05.352903 25745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:05.354151 25745 hybrid_clock.cc:648] HybridClock initialized: now 1786315865354064 us; error 51 us; skew 500 ppm
I20260809 22:51:05.354491 25745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:51:05.356915 25745 webserver.cc:460] Webserver started at http://127.24.136.2:34483/ using document root <none> and password file <none>
I20260809 22:51:05.357654 25745 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:05.357755 25745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:05.358067 25745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:05.361191 25745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "78725243e5b542eb93e288a1c603e2ab"
format_stamp: "Formatted at 2026-08-09 22:51:05 on dist-test-slave-pww0"
server_key: "93dcab8574a7840b984d240c9336eefd"
server_key_iv: "d542d3d000b86f32fac23efec9cbcc68"
server_key_version: "encryptionkey@0"
I20260809 22:51:05.362035 25745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "78725243e5b542eb93e288a1c603e2ab"
format_stamp: "Formatted at 2026-08-09 22:51:05 on dist-test-slave-pww0"
server_key: "93dcab8574a7840b984d240c9336eefd"
server_key_iv: "d542d3d000b86f32fac23efec9cbcc68"
server_key_version: "encryptionkey@0"
I20260809 22:51:05.367641 25745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 22:51:05.371780 25767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:05.373458 25745 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:51:05.373662 25745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "78725243e5b542eb93e288a1c603e2ab"
format_stamp: "Formatted at 2026-08-09 22:51:05 on dist-test-slave-pww0"
server_key: "93dcab8574a7840b984d240c9336eefd"
server_key_iv: "d542d3d000b86f32fac23efec9cbcc68"
server_key_version: "encryptionkey@0"
I20260809 22:51:05.373924 25745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:05.404296 25745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:05.405285 25745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:05.405607 25745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:05.406716 25745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:05.408384 25745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:05.408490 25745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:05.408617 25745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:05.408689 25745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:05.441280 25745 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:45489
I20260809 22:51:05.441334 25958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:45489 every 8 connection(s)
I20260809 22:51:05.442703 25745 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:51:05.445555 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 25745
I20260809 22:51:05.445745 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:51:05.446197 25120 external_mini_cluster.cc:1428] Setting key b9f681af5e8dae21b2670e26b91cc4d7
I20260809 22:51:05.449446 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42059
--builtin_ntp_servers=127.24.136.20:45497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:51:05.457239 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:05.445492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:45441 (local address 127.24.136.62:42059)
0809 22:51:05.445726 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:05.445733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:05.447260 (+ 1527us) server_negotiation.cc:407] Connection header received
0809 22:51:05.449561 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:05.449567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:05.449689 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:05.449789 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:05.451257 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.452151 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.454271 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.454696 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.454804 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:05.456457 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:05.456462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:05.456466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:05.456487 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:05.456524 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:05.456540 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:05.456681 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:05.456910 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 22:51:05.457002 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:51:05.458729 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:05.446006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42059 (local address 127.24.136.2:45441)
0809 22:51:05.447078 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:05.447126 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:05.449223 (+ 2097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:05.450070 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:05.450082 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:05.450205 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:05.451057 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:05.451083 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.452325 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.452330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:05.454089 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:05.454105 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.454935 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.454941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:05.455231 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:05.456046 (+ 815us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:51:05.456050 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:05.456108 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:51:05.456223 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:05.456676 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:05.456686 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:05.456695 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:51:05.456887 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 22:51:05.457327 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":329,"threads_started":1}
I20260809 22:51:05.460695 25959 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42059
I20260809 22:51:05.461046 25959 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:05.461983 25959 heartbeater.cc:511] Master 127.24.136.62:42059 requested a full tablet report, sending...
I20260809 22:51:05.463771 25394 ts_manager.cc:194] Registered new tserver with Master: 78725243e5b542eb93e288a1c603e2ab (127.24.136.2:45489)
I20260809 22:51:05.464632 25394 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.2:45441
W20260809 22:51:05.810930 25967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:05.811251 25967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:05.811393 25967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:05.819329 25967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:05.819495 25967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:05.819651 25967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:51:05.829566 25967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:45497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:42059
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:05.831020 25967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:05.833057 25967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:05.846560 25978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:05.846529 25977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:05.847852 25981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:05.847880 25967 server_base.cc:1034] running on GCE node
I20260809 22:51:05.848742 25967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:05.850111 25967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:05.851374 25967 hybrid_clock.cc:648] HybridClock initialized: now 1786315865851310 us; error 49 us; skew 500 ppm
I20260809 22:51:05.851820 25967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:51:05.854593 25967 webserver.cc:460] Webserver started at http://127.24.136.3:33299/ using document root <none> and password file <none>
I20260809 22:51:05.855345 25967 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:05.855477 25967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:05.855811 25967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:05.858917 25967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "36ab732681e540e9886dc1021f934473"
format_stamp: "Formatted at 2026-08-09 22:51:05 on dist-test-slave-pww0"
server_key: "03bc1795a15d5c938905d80bd0d27955"
server_key_iv: "fc6a703861b56cc4837c998cec91d309"
server_key_version: "encryptionkey@0"
I20260809 22:51:05.859812 25967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "36ab732681e540e9886dc1021f934473"
format_stamp: "Formatted at 2026-08-09 22:51:05 on dist-test-slave-pww0"
server_key: "03bc1795a15d5c938905d80bd0d27955"
server_key_iv: "fc6a703861b56cc4837c998cec91d309"
server_key_version: "encryptionkey@0"
I20260809 22:51:05.865437 25967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:51:05.869540 25992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:05.871253 25967 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 22:51:05.871423 25967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "36ab732681e540e9886dc1021f934473"
format_stamp: "Formatted at 2026-08-09 22:51:05 on dist-test-slave-pww0"
server_key: "03bc1795a15d5c938905d80bd0d27955"
server_key_iv: "fc6a703861b56cc4837c998cec91d309"
server_key_version: "encryptionkey@0"
I20260809 22:51:05.871701 25967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:05.888882 25967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:05.889772 25967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:05.890070 25967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:05.891168 25967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:05.892767 25967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:05.892880 25967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:05.892992 25967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:05.893062 25967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:05.927860 25967 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:34505
I20260809 22:51:05.927881 26193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:34505 every 8 connection(s)
I20260809 22:51:05.929271 25967 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:51:05.935060 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 25967
I20260809 22:51:05.935226 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:51:05.935787 25120 external_mini_cluster.cc:1428] Setting key 29963dbf8b7776b9a32ff221faf8537f
I20260809 22:51:05.945215 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:05.932859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:43601 (local address 127.24.136.62:42059)
0809 22:51:05.933198 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:05.933204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:05.934727 (+ 1523us) server_negotiation.cc:407] Connection header received
0809 22:51:05.937935 (+ 3208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:05.937941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:05.938037 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:05.938134 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:05.939527 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.940501 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.942226 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.942641 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.942759 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:05.944235 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:05.944241 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:05.944244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:05.944265 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:05.944298 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:05.944310 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:05.944416 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:05.944743 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 22:51:05.944852 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260809 22:51:05.946754 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:05.933349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:42059 (local address 127.24.136.3:43601)
0809 22:51:05.934519 (+ 1170us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:05.934559 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:05.937582 (+ 3023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:05.938422 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:05.938432 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:05.938535 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:05.939325 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:05.939352 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.940665 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.940669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:05.942064 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:05.942078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.942891 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.942900 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:05.943138 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:05.943874 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:51:05.943877 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:05.943922 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:51:05.944015 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:05.944464 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:05.944479 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:05.944490 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 22:51:05.944720 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 22:51:05.945209 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":406,"threads_started":1}
I20260809 22:51:05.948431 26194 heartbeater.cc:347] Connected to a master server at 127.24.136.62:42059
I20260809 22:51:05.948758 26194 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:05.949553 26194 heartbeater.cc:511] Master 127.24.136.62:42059 requested a full tablet report, sending...
I20260809 22:51:05.951231 25394 ts_manager.cc:194] Registered new tserver with Master: 36ab732681e540e9886dc1021f934473 (127.24.136.3:34505)
I20260809 22:51:05.952279 25394 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.3:43601
I20260809 22:51:05.953126 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:51:05.966071 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:05.958562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.24.136.62:42059)
0809 22:51:05.958876 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:05.958883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:05.959340 (+ 457us) server_negotiation.cc:407] Connection header received
0809 22:51:05.959563 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:05.959567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:05.959678 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:05.959761 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:05.960436 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.961167 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.963247 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:05.963762 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:05.963870 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:05.965275 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:05.965281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:05.965284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:05.965306 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:05.965344 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:05.965357 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:05.965495 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:05.965683 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 22:51:05.965739 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260809 22:51:05.968911 25735 heartbeater.cc:503] Master 127.24.136.62:42059 was elected leader, sending a full tablet report...
I20260809 22:51:05.971118 25394 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:51:05.978557 25394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:51:06.021715 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:05.995628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.24.136.1:45673)
0809 22:51:06.011208 (+ 15580us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:06.011219 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:06.012568 (+ 1349us) server_negotiation.cc:407] Connection header received
0809 22:51:06.012720 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:06.012728 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:06.012967 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:06.013168 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:06.014272 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.015819 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.018891 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.020439 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.020565 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.020874 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:06.021134 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 22:51:06.021370 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15367,"thread_start_us":15201,"threads_started":1}
I20260809 22:51:06.021699 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:05.995273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.0.0.1:56394)
0809 22:51:06.012023 (+ 16750us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.012084 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:06.012340 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:06.013466 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:06.013472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:06.013506 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:06.014032 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.014046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.015960 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.015965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.018642 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.018659 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.020648 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.020656 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.020865 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.020924 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 22:51:06.021047 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 22:51:06.021263 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16596,"mutex_wait_us":115,"spinlock_wait_cycles":28672,"thread_start_us":16078,"threads_started":1}
I20260809 22:51:06.022678 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:05.995295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55904 (local address 127.24.136.2:45489)
0809 22:51:06.006918 (+ 11623us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:06.006943 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:06.012265 (+ 5322us) server_negotiation.cc:407] Connection header received
0809 22:51:06.012572 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:06.012579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:06.012829 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:06.013032 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:06.014155 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.015612 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.018916 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.021125 (+ 2209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.021270 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.021509 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:06.022154 (+ 645us) server_negotiation.cc:299] Negotiation successful
0809 22:51:06.022343 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11394,"thread_start_us":11251,"threads_started":1}
I20260809 22:51:06.023289 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.014934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:34505 (local address 127.0.0.1:44624)
0809 22:51:06.015534 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.015553 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:06.015686 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:06.016301 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:06.016305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:06.016328 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:06.016808 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.016818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.018166 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.018171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.020130 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.020148 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.022459 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.022466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.022650 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.022683 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:51:06.022779 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:51:06.022911 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":21,"thread_start_us":297,"threads_started":1}
I20260809 22:51:06.023362 26217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:05.994804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:45489 (local address 127.0.0.1:55904)
0809 22:51:06.012023 (+ 17219us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.012135 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:06.012339 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:06.013323 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:06.013330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:06.013376 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:06.013921 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.013939 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.015792 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.015797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.018642 (+ 2845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.018659 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.021662 (+ 3003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.021669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.021916 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.021950 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:51:06.022882 (+ 932us) client_negotiation.cc:240] Negotiation successful
0809 22:51:06.023002 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16893,"mutex_wait_us":196,"spinlock_wait_cycles":8320,"thread_start_us":16865,"threads_started":1}
I20260809 22:51:06.023615 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.012721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.24.136.3:34505)
0809 22:51:06.014888 (+ 2167us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:06.014897 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:06.015653 (+ 756us) server_negotiation.cc:407] Connection header received
0809 22:51:06.015829 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:06.015838 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:06.016095 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:06.016280 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:06.016988 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.018001 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.020361 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.021955 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.022078 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.022331 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:06.022946 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 22:51:06.023116 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1991,"thread_start_us":1829,"threads_started":1}
I20260809 22:51:06.026211 25617 tablet_service.cc:1467] Processing CreateTablet for tablet 3d76644393fa464c950210794c172958 (DEFAULT_TABLE table=test-table [id=5008dfa120ca4ae09fa7afc7cf8ada2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:06.027312 26070 tablet_service.cc:1467] Processing CreateTablet for tablet 3d76644393fa464c950210794c172958 (DEFAULT_TABLE table=test-table [id=5008dfa120ca4ae09fa7afc7cf8ada2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:06.027925 25617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d76644393fa464c950210794c172958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:06.028844 26070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d76644393fa464c950210794c172958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:06.030831 25859 tablet_service.cc:1467] Processing CreateTablet for tablet 3d76644393fa464c950210794c172958 (DEFAULT_TABLE table=test-table [id=5008dfa120ca4ae09fa7afc7cf8ada2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:06.032146 25859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d76644393fa464c950210794c172958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:06.043903 26225 tablet_bootstrap.cc:492] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: Bootstrap starting.
I20260809 22:51:06.047339 26225 tablet_bootstrap.cc:654] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:06.047956 26227 tablet_bootstrap.cc:492] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c: Bootstrap starting.
I20260809 22:51:06.048468 26226 tablet_bootstrap.cc:492] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: Bootstrap starting.
I20260809 22:51:06.050704 26225 log.cc:824] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:06.051430 26227 tablet_bootstrap.cc:654] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:06.052340 26226 tablet_bootstrap.cc:654] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:06.053259 26227 log.cc:824] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:06.053483 26226 log.cc:824] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:06.055369 26225 tablet_bootstrap.cc:492] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: No bootstrap required, opened a new log
I20260809 22:51:06.055765 26225 ts_tablet_manager.cc:1397] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260809 22:51:06.056181 26227 tablet_bootstrap.cc:492] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c: No bootstrap required, opened a new log
I20260809 22:51:06.056456 26227 ts_tablet_manager.cc:1397] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 22:51:06.064694 26227 raft_consensus.cc:348] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } }
I20260809 22:51:06.065117 26227 raft_consensus.cc:374] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:06.065214 26227 raft_consensus.cc:729] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f917889777ac4108a1fea1a43050c06c, State: Initialized, Role: FOLLOWER
I20260809 22:51:06.065601 26226 tablet_bootstrap.cc:492] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: No bootstrap required, opened a new log
I20260809 22:51:06.065932 26226 ts_tablet_manager.cc:1397] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.010s
I20260809 22:51:06.065892 26227 consensus_queue.cc:260] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } }
I20260809 22:51:06.067534 26225 raft_consensus.cc:348] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } }
I20260809 22:51:06.068037 26225 raft_consensus.cc:374] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:06.068132 26225 raft_consensus.cc:729] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78725243e5b542eb93e288a1c603e2ab, State: Initialized, Role: FOLLOWER
I20260809 22:51:06.068993 26227 ts_tablet_manager.cc:1428] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260809 22:51:06.071405 26225 consensus_queue.cc:260] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } }
I20260809 22:51:06.072950 25959 heartbeater.cc:503] Master 127.24.136.62:42059 was elected leader, sending a full tablet report...
I20260809 22:51:06.073489 26225 ts_tablet_manager.cc:1428] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: Time spent starting tablet: real 0.018s user 0.013s sys 0.000s
I20260809 22:51:06.076145 26226 raft_consensus.cc:348] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } }
I20260809 22:51:06.076606 26226 raft_consensus.cc:374] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:06.076717 26226 raft_consensus.cc:729] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36ab732681e540e9886dc1021f934473, State: Initialized, Role: FOLLOWER
I20260809 22:51:06.077395 26226 consensus_queue.cc:260] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } }
I20260809 22:51:06.078536 26194 heartbeater.cc:503] Master 127.24.136.62:42059 was elected leader, sending a full tablet report...
I20260809 22:51:06.079154 26226 ts_tablet_manager.cc:1428] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: Time spent starting tablet: real 0.013s user 0.008s sys 0.007s
W20260809 22:51:06.181439 26196 tablet.cc:2369] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:51:06.194844 25961 tablet.cc:2369] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:51:06.195840 25739 tablet.cc:2369] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:51:06.258512 26240 raft_consensus.cc:482] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:51:06.258894 26240 raft_consensus.cc:504] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } }
I20260809 22:51:06.260821 26240 leader_election.cc:288] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78725243e5b542eb93e288a1c603e2ab (127.24.136.2:45489), f917889777ac4108a1fea1a43050c06c (127.24.136.1:45673)
I20260809 22:51:06.266846 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.261674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:44701 (local address 127.24.136.1:45673)
0809 22:51:06.261916 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:06.261923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:06.261944 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:51:06.262019 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:06.262023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:06.262103 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:06.262187 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:06.262631 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.263350 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.265097 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.265915 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.266087 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.266165 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:06.266457 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 22:51:06.266589 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 22:51:06.266891 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.261192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.24.136.3:44701)
0809 22:51:06.261438 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.261459 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:06.261607 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:06.262269 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:06.262274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:06.262304 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:06.262454 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.262462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.263539 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.263543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.264912 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.264926 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.266084 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.266090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.266283 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.266310 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 22:51:06.266395 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 22:51:06.266554 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":1}
I20260809 22:51:06.267283 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.261418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:39213 (local address 127.24.136.2:45489)
0809 22:51:06.261624 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:06.261630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:06.262177 (+ 547us) server_negotiation.cc:407] Connection header received
0809 22:51:06.262357 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:06.262361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:06.262432 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:06.262508 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:06.263067 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.263834 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.265592 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.266356 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.266518 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.266588 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:06.266885 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 22:51:06.267043 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:51:06.267316 26242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.261254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:45489 (local address 127.24.136.3:39213)
0809 22:51:06.262051 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.262066 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:06.262212 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:06.262553 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:06.262558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:06.262581 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:06.262898 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.262908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.263997 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.264001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.265384 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:06.265398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.266547 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.266553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:06.266725 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.266750 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:51:06.266829 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 22:51:06.266979 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":528,"threads_started":1}
I20260809 22:51:06.268100 25660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d76644393fa464c950210794c172958" candidate_uuid: "36ab732681e540e9886dc1021f934473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f917889777ac4108a1fea1a43050c06c" is_pre_election: true
I20260809 22:51:06.268362 25890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d76644393fa464c950210794c172958" candidate_uuid: "36ab732681e540e9886dc1021f934473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78725243e5b542eb93e288a1c603e2ab" is_pre_election: true
I20260809 22:51:06.268647 25660 raft_consensus.cc:2454] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36ab732681e540e9886dc1021f934473 in term 0.
I20260809 22:51:06.268899 25890 raft_consensus.cc:2454] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36ab732681e540e9886dc1021f934473 in term 0.
I20260809 22:51:06.269388 26001 leader_election.cc:302] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36ab732681e540e9886dc1021f934473, f917889777ac4108a1fea1a43050c06c; no voters:
I20260809 22:51:06.269902 26240 raft_consensus.cc:2781] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:51:06.270041 26240 raft_consensus.cc:482] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:51:06.270141 26240 raft_consensus.cc:3037] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:06.272514 26240 raft_consensus.cc:504] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } }
I20260809 22:51:06.273180 26240 leader_election.cc:288] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [CANDIDATE]: Term 1 election: Requested vote from peers 78725243e5b542eb93e288a1c603e2ab (127.24.136.2:45489), f917889777ac4108a1fea1a43050c06c (127.24.136.1:45673)
I20260809 22:51:06.273885 25660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d76644393fa464c950210794c172958" candidate_uuid: "36ab732681e540e9886dc1021f934473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f917889777ac4108a1fea1a43050c06c"
I20260809 22:51:06.273885 25890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d76644393fa464c950210794c172958" candidate_uuid: "36ab732681e540e9886dc1021f934473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78725243e5b542eb93e288a1c603e2ab"
I20260809 22:51:06.274139 25660 raft_consensus.cc:3037] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:06.274159 25890 raft_consensus.cc:3037] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:06.277614 25660 raft_consensus.cc:2454] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36ab732681e540e9886dc1021f934473 in term 1.
I20260809 22:51:06.277614 25890 raft_consensus.cc:2454] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36ab732681e540e9886dc1021f934473 in term 1.
I20260809 22:51:06.278339 25996 leader_election.cc:302] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 36ab732681e540e9886dc1021f934473, 78725243e5b542eb93e288a1c603e2ab, f917889777ac4108a1fea1a43050c06c; no voters:
I20260809 22:51:06.278734 26240 raft_consensus.cc:2781] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:06.279260 26240 raft_consensus.cc:686] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 1 LEADER]: Becoming Leader. State: Replica: 36ab732681e540e9886dc1021f934473, State: Running, Role: LEADER
I20260809 22:51:06.279886 26240 consensus_queue.cc:237] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } }
I20260809 22:51:06.285004 25394 catalog_manager.cc:5495] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36ab732681e540e9886dc1021f934473 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "36ab732681e540e9886dc1021f934473" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36ab732681e540e9886dc1021f934473" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 34505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 } health_report { overall_health: UNKNOWN } } }
I20260809 22:51:06.310703 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.304942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44626 (local address 127.24.136.3:34505)
0809 22:51:06.305197 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:06.305220 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:06.305242 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:51:06.305381 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:06.305386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:06.305478 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:06.305625 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:06.306086 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.306832 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.308358 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.308799 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.308917 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.309556 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:06.310080 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:06.310258 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:06.310427 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 22:51:06.310474 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 22:51:06.319734 25890 raft_consensus.cc:1261] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [term 1 FOLLOWER]: Refusing update from remote peer 36ab732681e540e9886dc1021f934473: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:06.319730 25660 raft_consensus.cc:1261] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [term 1 FOLLOWER]: Refusing update from remote peer 36ab732681e540e9886dc1021f934473: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:06.320740 26243 consensus_queue.cc:1035] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f917889777ac4108a1fea1a43050c06c" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 45673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:51:06.321094 26240 consensus_queue.cc:1035] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78725243e5b542eb93e288a1c603e2ab" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 45489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:51:06.352523 25394 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:51:06.352803 25394 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:51:06.356866 25394 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9f677f4c5834504a5c55e3350b985b4: Sending DeleteTablet for 3 replicas of tablet 3d76644393fa464c950210794c172958
I20260809 22:51:06.358587 25859 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d76644393fa464c950210794c172958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:06 UTC) from {username='slave'} at 127.0.0.1:55904
I20260809 22:51:06.358587 25617 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d76644393fa464c950210794c172958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:06 UTC) from {username='slave'} at 127.0.0.1:56394
I20260809 22:51:06.358587 26070 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d76644393fa464c950210794c172958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:06 UTC) from {username='slave'} at 127.0.0.1:44624
I20260809 22:51:06.360479 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 25510
I20260809 22:51:06.360561 26267 tablet_replica.cc:330] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: stopping tablet replica
I20260809 22:51:06.368059 26266 tablet_replica.cc:330] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: stopping tablet replica
I20260809 22:51:06.368785 26266 raft_consensus.cc:2229] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:51:06.369382 26266 raft_consensus.cc:2258] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:06.372354 26267 raft_consensus.cc:2229] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:51:06.373395 26267 raft_consensus.cc:2258] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:06.375895 26265 tablet_replica.cc:330] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c: stopping tablet replica
I20260809 22:51:06.376643 26265 raft_consensus.cc:2229] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:51:06.377277 26265 raft_consensus.cc:2258] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:06.377766 26267 ts_tablet_manager.cc:1905] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:06.378038 26266 ts_tablet_manager.cc:1905] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:06.386755 26266 ts_tablet_manager.cc:1918] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:51:06.386991 26266 log.cc:1196] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/3d76644393fa464c950210794c172958
I20260809 22:51:06.386852 26267 ts_tablet_manager.cc:1918] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:51:06.387336 26267 log.cc:1196] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/3d76644393fa464c950210794c172958
I20260809 22:51:06.387625 26266 ts_tablet_manager.cc:1939] T 3d76644393fa464c950210794c172958 P 78725243e5b542eb93e288a1c603e2ab: Deleting consensus metadata
I20260809 22:51:06.388116 26267 ts_tablet_manager.cc:1939] T 3d76644393fa464c950210794c172958 P 36ab732681e540e9886dc1021f934473: Deleting consensus metadata
I20260809 22:51:06.388540 26265 ts_tablet_manager.cc:1905] T 3d76644393fa464c950210794c172958 P f917889777ac4108a1fea1a43050c06c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:06.390170 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.364615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56402 (local address 127.24.136.1:45673)
0809 22:51:06.364836 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:06.364843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:06.370323 (+ 5480us) server_negotiation.cc:407] Connection header received
0809 22:51:06.370645 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:06.370651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:06.370790 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:06.370863 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:06.371596 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.372906 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.376199 (+ 3293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.376984 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.377097 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.388151 (+ 11054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:06.388167 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:06.388187 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:06.389043 (+ 856us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:06.389127 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:06.389161 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:06.389293 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:06.389611 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 22:51:06.389756 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:51:06.390347 25365 catalog_manager.cc:4841] TS 36ab732681e540e9886dc1021f934473 (127.24.136.3:34505): tablet 3d76644393fa464c950210794c172958 (table test-table [id=5008dfa120ca4ae09fa7afc7cf8ada2a]) successfully deleted
I20260809 22:51:06.391542 25728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:06.395952 25365 catalog_manager.cc:4841] TS 78725243e5b542eb93e288a1c603e2ab (127.24.136.2:45489): tablet 3d76644393fa464c950210794c172958 (table test-table [id=5008dfa120ca4ae09fa7afc7cf8ada2a]) successfully deleted
I20260809 22:51:06.486545 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 25510
I20260809 22:51:06.510177 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 25745
W20260809 22:51:06.510263 25372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.136.1:45673 (error 108)
I20260809 22:51:06.512498 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.511956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.0.0.1:56406)
0809 22:51:06.512156 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.512301 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 22:51:06.512820 25372 catalog_manager.cc:4584] TS f917889777ac4108a1fea1a43050c06c (127.24.136.1:45673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d76644393fa464c950210794c172958: Network error: Client connection negotiation failed: client connection to 127.24.136.1:45673: connect: Connection refused (error 111)
I20260809 22:51:06.518597 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.510996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.24.136.2:45489)
0809 22:51:06.511200 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:06.511207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:06.511230 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:51:06.511434 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:06.511439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:06.511598 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:06.511706 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:06.512339 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.513564 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.514985 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.515671 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.515808 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.516883 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:06.516896 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:06.516915 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:06.517746 (+ 831us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:06.517814 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:06.517849 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:06.517966 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:06.518091 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 22:51:06.518220 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:51:06.519443 25953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:06.566795 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.566166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.0.0.1:56416)
0809 22:51:06.566462 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.566606 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260809 22:51:06.568194 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.567729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.0.0.1:56428)
0809 22:51:06.567905 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.568007 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 22:51:06.611126 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 25745
I20260809 22:51:06.628813 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.628186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.0.0.1:56440)
0809 22:51:06.628456 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.628590 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 22:51:06.630142 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.629706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.0.0.1:56442)
0809 22:51:06.629890 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.629984 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 22:51:06.644733 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 25967
I20260809 22:51:06.653020 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.645493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44640 (local address 127.24.136.3:34505)
0809 22:51:06.645725 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:06.645731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:06.645764 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:51:06.645875 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:06.645881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:06.645993 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:06.646119 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:06.646506 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.647814 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.649263 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:06.649907 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:06.650059 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:06.651158 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:06.651173 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:06.651196 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:06.652118 (+ 922us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:06.652205 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:06.652234 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:06.652364 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:06.652501 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 22:51:06.652621 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 22:51:06.653796 26184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:06.711076 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.710452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.0.0.1:56458)
0809 22:51:06.710717 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.710847 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 22:51:06.712396 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.711891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.0.0.1:56472)
0809 22:51:06.712105 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.712199 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260809 22:51:06.777251 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 25967
I20260809 22:51:06.801424 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 25336
I20260809 22:51:06.802580 25452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:06.912787 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:06.911925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:45673 (local address 127.0.0.1:56482)
0809 22:51:06.912295 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:06.912408 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:45673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
I20260809 22:51:06.912869 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 25336
2026-08-09T22:51:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3102 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:07 dist-test-slave-pww0 krb5kdc[26288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:07 dist-test-slave-pww0 krb5kdc[26288](info): set up 2 sockets
Aug 09 22:51:07 dist-test-slave-pww0 krb5kdc[26288](info): commencing operation
krb5kdc: starting...
W20260809 22:51:08.204239 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:08 dist-test-slave-pww0 krb5kdc[26288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:51:08.531658 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:37809
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:37809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:08.890002 26309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:08.890375 26309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:08.890522 26309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:08.898069 26309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:08.898221 26309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:08.898309 26309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:08.898388 26309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:08.898449 26309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:08.909184 26309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:37809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:37809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:08.910686 26309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:08.912164 26309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:08.924836 26325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:08.924866 26326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:08.924880 26328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:08.925904 26309 server_base.cc:1034] running on GCE node
I20260809 22:51:08.926831 26309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:08.928509 26309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:08.929736 26309 hybrid_clock.cc:648] HybridClock initialized: now 1786315868929670 us; error 62 us; skew 500 ppm
I20260809 22:51:08.930133 26309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:08 dist-test-slave-pww0 krb5kdc[26288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315868, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:08.933816 26309 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:51:08.935767 26309 webserver.cc:460] Webserver started at http://127.24.136.62:39645/ using document root <none> and password file <none>
I20260809 22:51:08.936555 26309 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:08.936695 26309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:08.937186 26309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:08.940788 26309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "63e790c3f0ef42129b38c0f47699d5e4"
format_stamp: "Formatted at 2026-08-09 22:51:08 on dist-test-slave-pww0"
server_key: "29cb0c914d45ad37aad91578ef71a707"
server_key_iv: "b56e6324cb1b36915329b00ae1b853e3"
server_key_version: "encryptionkey@0"
I20260809 22:51:08.941740 26309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "63e790c3f0ef42129b38c0f47699d5e4"
format_stamp: "Formatted at 2026-08-09 22:51:08 on dist-test-slave-pww0"
server_key: "29cb0c914d45ad37aad91578ef71a707"
server_key_iv: "b56e6324cb1b36915329b00ae1b853e3"
server_key_version: "encryptionkey@0"
I20260809 22:51:08.948292 26309 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 22:51:08.952396 26340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:08.954133 26309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:51:08.954346 26309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "63e790c3f0ef42129b38c0f47699d5e4"
format_stamp: "Formatted at 2026-08-09 22:51:08 on dist-test-slave-pww0"
server_key: "29cb0c914d45ad37aad91578ef71a707"
server_key_iv: "b56e6324cb1b36915329b00ae1b853e3"
server_key_version: "encryptionkey@0"
I20260809 22:51:08.954563 26309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:08.973959 26309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:08.979095 26309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:08.979614 26309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:08.999984 26309 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:37809
I20260809 22:51:08.999994 26428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:37809 every 8 connection(s)
I20260809 22:51:09.001556 26309 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:09.004586 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 26309
I20260809 22:51:09.004810 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:09.005409 25120 external_mini_cluster.cc:1428] Setting key 03e126bb676f871d80f33f52c55b8d2d
I20260809 22:51:09.006625 26430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:51:09 dist-test-slave-pww0 krb5kdc[26288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:09.022287 26430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4: Bootstrap starting.
I20260809 22:51:09.025933 26430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:09.027102 26430 log.cc:824] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:09.030381 26430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4: No bootstrap required, opened a new log
I20260809 22:51:09.031776 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:09.007581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.24.136.62:37809)
0809 22:51:09.008420 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:09.008434 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:09.008479 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:51:09.010180 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:09.010207 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:09.010698 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:09.011102 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:09.013282 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:09.014930 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:09.016918 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:09.017785 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:09.017969 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:09.023381 (+ 5412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:09.023392 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:09.023409 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:09.023513 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:09.026813 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:09.027821 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:09.027832 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:09.027839 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:09.028036 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:09.028479 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:09.028483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:09.028485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:09.029083 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:09.029382 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:09.029867 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 22:51:09.030337 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":384,"threads_started":1}
I20260809 22:51:09.039429 26430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e790c3f0ef42129b38c0f47699d5e4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37809 } }
I20260809 22:51:09.040027 26430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:09.040140 26430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e790c3f0ef42129b38c0f47699d5e4, State: Initialized, Role: FOLLOWER
I20260809 22:51:09.040786 26430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e790c3f0ef42129b38c0f47699d5e4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37809 } }
I20260809 22:51:09.041019 26430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:09.041162 26430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:09.041328 26430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:09.043783 26430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e790c3f0ef42129b38c0f47699d5e4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37809 } }
I20260809 22:51:09.044479 26430 leader_election.cc:302] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63e790c3f0ef42129b38c0f47699d5e4; no voters:
I20260809 22:51:09.045218 26430 leader_election.cc:288] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:09.045382 26441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:09.046068 26441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [term 1 LEADER]: Becoming Leader. State: Replica: 63e790c3f0ef42129b38c0f47699d5e4, State: Running, Role: LEADER
I20260809 22:51:09.046660 26441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e790c3f0ef42129b38c0f47699d5e4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37809 } }
I20260809 22:51:09.047382 26430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:09.049633 26443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63e790c3f0ef42129b38c0f47699d5e4. Latest consensus state: current_term: 1 leader_uuid: "63e790c3f0ef42129b38c0f47699d5e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e790c3f0ef42129b38c0f47699d5e4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37809 } } }
I20260809 22:51:09.050103 26443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:09.050835 26442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63e790c3f0ef42129b38c0f47699d5e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63e790c3f0ef42129b38c0f47699d5e4" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 37809 } } }
I20260809 22:51:09.051146 26442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:09.052052 26454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:09.057201 26454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:09.067266 26454 catalog_manager.cc:1347] Generated new cluster ID: 90e34ae95ae44b298ebc2aeeeba7cc34
I20260809 22:51:09.067413 26454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:09.092311 26454 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:09.094481 26454 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:09.117084 26454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4: Generated new TSK 0
I20260809 22:51:09.118386 26454 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.136.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
Entry for principal kudu/127.24.136.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab.
I20260809 22:51:09.199941 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37809
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:09.560242 26475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:09.560546 26475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:09.560691 26475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:09.568125 26475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:09.568259 26475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:09.568421 26475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:51:09.578471 26475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.1.keytab
--principal=kudu/127.24.136.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:09.579897 26475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:09.581302 26475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:09.595269 26483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:09.595198 26490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:09.595243 26484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:09.596110 26475 server_base.cc:1034] running on GCE node
I20260809 22:51:09.596778 26475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:09.597800 26475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:09.599069 26475 hybrid_clock.cc:648] HybridClock initialized: now 1786315869598992 us; error 54 us; skew 500 ppm
I20260809 22:51:09.599498 26475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:09 dist-test-slave-pww0 krb5kdc[26288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315869, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:09.602829 26475 init.cc:377] Logged in from keytab as kudu/127.24.136.1@KRBTEST.COM (short username kudu)
I20260809 22:51:09.604487 26475 webserver.cc:460] Webserver started at http://127.24.136.1:35535/ using document root <none> and password file <none>
I20260809 22:51:09.605229 26475 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:09.605360 26475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:09.605676 26475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:09.608839 26475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "36d84ffbe6d2425a8e5fb27a974f10bf"
format_stamp: "Formatted at 2026-08-09 22:51:09 on dist-test-slave-pww0"
server_key: "b84482f004dfad3cd8be441319b73d51"
server_key_iv: "b6cb9baa935853808b1a8fc679246b41"
server_key_version: "encryptionkey@0"
I20260809 22:51:09.609665 26475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "36d84ffbe6d2425a8e5fb27a974f10bf"
format_stamp: "Formatted at 2026-08-09 22:51:09 on dist-test-slave-pww0"
server_key: "b84482f004dfad3cd8be441319b73d51"
server_key_iv: "b6cb9baa935853808b1a8fc679246b41"
server_key_version: "encryptionkey@0"
I20260809 22:51:09.615304 26475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:51:09.619369 26504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:09.621172 26475 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:51:09.621363 26475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "36d84ffbe6d2425a8e5fb27a974f10bf"
format_stamp: "Formatted at 2026-08-09 22:51:09 on dist-test-slave-pww0"
server_key: "b84482f004dfad3cd8be441319b73d51"
server_key_iv: "b6cb9baa935853808b1a8fc679246b41"
server_key_version: "encryptionkey@0"
I20260809 22:51:09.621572 26475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:09.667143 26475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:09.671790 26475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:09.672103 26475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:09.673137 26475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:09.674552 26475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:09.674636 26475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:09.674764 26475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:09.674820 26475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:09.706143 26475 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:38053
I20260809 22:51:09.706180 26704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:38053 every 8 connection(s)
I20260809 22:51:09.707499 26475 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:51:09.713851 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 26475
I20260809 22:51:09.714058 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:51:09.714617 25120 external_mini_cluster.cc:1428] Setting key 926ea8da2ef58716f2946e39339d177b
Aug 09 22:51:09 dist-test-slave-pww0 krb5kdc[26288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315869, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.1@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:09.730052 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:09.710083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:43529 (local address 127.24.136.62:37809)
0809 22:51:09.710580 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:09.710586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:09.711643 (+ 1057us) server_negotiation.cc:407] Connection header received
0809 22:51:09.713554 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:09.713559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:09.713636 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:09.713737 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:09.715795 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:09.716588 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:09.718277 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:09.718712 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:09.718835 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:09.722713 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:09.722719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:09.722722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:09.722761 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:09.725447 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:09.726598 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:09.726604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:09.726607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:09.726878 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:09.727524 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:09.727530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:09.727533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:09.727969 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:09.728140 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:09.729183 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0809 22:51:09.729591 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":216,"threads_started":1}
I20260809 22:51:09.731050 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:09.710647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:37809 (local address 127.24.136.1:43529)
0809 22:51:09.711410 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:09.711476 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:09.713253 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:09.714037 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:09.714050 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:09.714836 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:09.715624 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:09.715647 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:09.716770 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:09.716775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:09.718105 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:09.718120 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:09.718997 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:09.719007 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:09.719239 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:09.719967 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:09.719993 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:09.722460 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:09.725625 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:09.725635 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:09.725654 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:09.726415 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:09.727036 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:09.727040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:09.727043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:09.727341 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:09.728128 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:09.728136 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:09.728612 (+ 476us) client_negotiation.cc:769] Sending connection context
0809 22:51:09.728921 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 22:51:09.729505 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":301,"threads_started":1}
I20260809 22:51:09.733042 26706 heartbeater.cc:347] Connected to a master server at 127.24.136.62:37809
I20260809 22:51:09.733376 26706 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:09.734187 26706 heartbeater.cc:511] Master 127.24.136.62:37809 requested a full tablet report, sending...
I20260809 22:51:09.737193 26372 ts_manager.cc:194] Registered new tserver with Master: 36d84ffbe6d2425a8e5fb27a974f10bf (127.24.136.1:38053)
I20260809 22:51:09.738776 26372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.1@KRBTEST.COM'} at 127.24.136.1:43529
WARNING: no policy specified for kudu/127.24.136.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
Entry for principal kudu/127.24.136.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab.
I20260809 22:51:09.790796 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37809
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:10.140516 26717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:10.140844 26717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:10.140982 26717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:10.148716 26717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:10.148864 26717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:10.149012 26717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:51:10.159690 26717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.2.keytab
--principal=kudu/127.24.136.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:10.161084 26717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:10.162460 26717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:10.176549 26733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:10.176515 26734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:10.176463 26737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:10.177347 26717 server_base.cc:1034] running on GCE node
I20260809 22:51:10.178026 26717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:10.179076 26717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:10.180320 26717 hybrid_clock.cc:648] HybridClock initialized: now 1786315870180260 us; error 72 us; skew 500 ppm
I20260809 22:51:10.180698 26717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:10 dist-test-slave-pww0 krb5kdc[26288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315870, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:10.184212 26717 init.cc:377] Logged in from keytab as kudu/127.24.136.2@KRBTEST.COM (short username kudu)
I20260809 22:51:10.185830 26717 webserver.cc:460] Webserver started at http://127.24.136.2:46813/ using document root <none> and password file <none>
I20260809 22:51:10.186569 26717 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:10.186698 26717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:10.187019 26717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:10.190291 26717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "9c74d9d61e624029a74085eef195abca"
format_stamp: "Formatted at 2026-08-09 22:51:10 on dist-test-slave-pww0"
server_key: "00c62dbe415c63ea57185a61ab67726c"
server_key_iv: "32f44ddc86d5e4872c404354534d97cc"
server_key_version: "encryptionkey@0"
I20260809 22:51:10.191202 26717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "9c74d9d61e624029a74085eef195abca"
format_stamp: "Formatted at 2026-08-09 22:51:10 on dist-test-slave-pww0"
server_key: "00c62dbe415c63ea57185a61ab67726c"
server_key_iv: "32f44ddc86d5e4872c404354534d97cc"
server_key_version: "encryptionkey@0"
I20260809 22:51:10.196846 26717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:51:10.200606 26751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:10.202030 26717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:10.202214 26717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "9c74d9d61e624029a74085eef195abca"
format_stamp: "Formatted at 2026-08-09 22:51:10 on dist-test-slave-pww0"
server_key: "00c62dbe415c63ea57185a61ab67726c"
server_key_iv: "32f44ddc86d5e4872c404354534d97cc"
server_key_version: "encryptionkey@0"
I20260809 22:51:10.202415 26717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:10.219743 26717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:10.224646 26717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:10.224980 26717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:10.226083 26717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:10.227589 26717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:10.227671 26717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:10.227799 26717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:10.227854 26717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:10.259097 26717 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:44303
I20260809 22:51:10.259195 26940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:44303 every 8 connection(s)
I20260809 22:51:10.260548 26717 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:51:10.263068 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 26717
I20260809 22:51:10.263270 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:51:10.263888 25120 external_mini_cluster.cc:1428] Setting key 2aec07946b7649c07d32704b814d5846
Aug 09 22:51:10 dist-test-slave-pww0 krb5kdc[26288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315870, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.2@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:10.282122 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.263265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:51589 (local address 127.24.136.62:37809)
0809 22:51:10.263789 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:10.263795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:10.266915 (+ 3120us) server_negotiation.cc:407] Connection header received
0809 22:51:10.266976 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:10.266980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:10.267049 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:10.267147 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:10.268917 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.269655 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.271322 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.271820 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.271944 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.275773 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:10.275779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:10.275782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:10.275824 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:10.278618 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:10.279527 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:10.279532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:10.279535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:10.279688 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:10.280181 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:10.280185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:10.280187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:10.280493 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:10.280657 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:10.281459 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 22:51:10.281721 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":252,"threads_started":1}
I20260809 22:51:10.283170 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.263891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:37809 (local address 127.24.136.2:51589)
0809 22:51:10.264788 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:10.264838 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:10.266599 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:10.267449 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:10.267488 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:10.268090 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:10.268707 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.268727 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.269840 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.269844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.271152 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.271166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.272076 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.272083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.272341 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.273062 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:10.273091 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:10.275446 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:10.278826 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:10.278837 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:10.278853 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:10.279347 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:10.279815 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:10.279820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:10.279822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:10.280039 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:10.280637 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:10.280656 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:10.281046 (+ 390us) client_negotiation.cc:769] Sending connection context
0809 22:51:10.281314 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 22:51:10.281918 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":336,"threads_started":1}
I20260809 22:51:10.284937 26941 heartbeater.cc:347] Connected to a master server at 127.24.136.62:37809
I20260809 22:51:10.285281 26941 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:10.286083 26941 heartbeater.cc:511] Master 127.24.136.62:37809 requested a full tablet report, sending...
I20260809 22:51:10.287837 26372 ts_manager.cc:194] Registered new tserver with Master: 9c74d9d61e624029a74085eef195abca (127.24.136.2:44303)
I20260809 22:51:10.288911 26372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.2@KRBTEST.COM'} at 127.24.136.2:51589
WARNING: no policy specified for kudu/127.24.136.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
Entry for principal kudu/127.24.136.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab.
I20260809 22:51:10.335052 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37809
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:10.691978 26953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:10.692320 26953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:10.692454 26953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:10.699945 26953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:10.700089 26953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:10.700244 26953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:51:10.713260 26953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.3.keytab
--principal=kudu/127.24.136.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:37809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:10.714625 26953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:10.716078 26953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:10.729735 26962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:10.729843 26960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:10.729804 26958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:10.730690 26953 server_base.cc:1034] running on GCE node
I20260809 22:51:10.731367 26953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:10.732440 26953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:10.733646 26953 hybrid_clock.cc:648] HybridClock initialized: now 1786315870733590 us; error 56 us; skew 500 ppm
I20260809 22:51:10.734002 26953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:10 dist-test-slave-pww0 krb5kdc[26288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315870, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:10.737381 26953 init.cc:377] Logged in from keytab as kudu/127.24.136.3@KRBTEST.COM (short username kudu)
I20260809 22:51:10.739138 26953 webserver.cc:460] Webserver started at http://127.24.136.3:37389/ using document root <none> and password file <none>
I20260809 22:51:10.739934 26953 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:10.740061 26953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:10.740387 26953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:10.742553 26706 heartbeater.cc:503] Master 127.24.136.62:37809 was elected leader, sending a full tablet report...
I20260809 22:51:10.743788 26953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "014a331cee2c469783e009e5b6fcc98f"
format_stamp: "Formatted at 2026-08-09 22:51:10 on dist-test-slave-pww0"
server_key: "0f28a72dcdcfed18ff9b44f5bf5a4493"
server_key_iv: "9cac3e762e9cd3783157124fd675c639"
server_key_version: "encryptionkey@0"
I20260809 22:51:10.744869 26953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "014a331cee2c469783e009e5b6fcc98f"
format_stamp: "Formatted at 2026-08-09 22:51:10 on dist-test-slave-pww0"
server_key: "0f28a72dcdcfed18ff9b44f5bf5a4493"
server_key_iv: "9cac3e762e9cd3783157124fd675c639"
server_key_version: "encryptionkey@0"
I20260809 22:51:10.750751 26953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:51:10.754782 26971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:10.756338 26953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:51:10.756518 26953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "014a331cee2c469783e009e5b6fcc98f"
format_stamp: "Formatted at 2026-08-09 22:51:10 on dist-test-slave-pww0"
server_key: "0f28a72dcdcfed18ff9b44f5bf5a4493"
server_key_iv: "9cac3e762e9cd3783157124fd675c639"
server_key_version: "encryptionkey@0"
I20260809 22:51:10.756734 26953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:10.774936 26953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:10.779763 26953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:10.780084 26953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:10.781112 26953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:10.782632 26953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:10.782713 26953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:10.782857 26953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:10.782914 26953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:10.815311 26953 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:39123
I20260809 22:51:10.815353 27163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:39123 every 8 connection(s)
I20260809 22:51:10.816666 26953 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:51:10.818557 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 26953
I20260809 22:51:10.818764 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:51:10.819236 25120 external_mini_cluster.cc:1428] Setting key 25028d07e7e5c732d5b16edf95706eb9
Aug 09 22:51:10 dist-test-slave-pww0 krb5kdc[26288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315870, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.3@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:10.838729 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.819488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53589 (local address 127.24.136.62:37809)
0809 22:51:10.820273 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:10.820277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:10.821186 (+ 909us) server_negotiation.cc:407] Connection header received
0809 22:51:10.823451 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:10.823477 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:10.823571 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:10.823661 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:10.825340 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.826185 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.828040 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.828507 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.828631 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.832322 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:10.832327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:10.832331 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:10.832378 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:10.835600 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:10.836377 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:10.836382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:10.836384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:10.836523 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:10.837056 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:10.837060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:10.837062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:10.837382 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:10.837502 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:10.838258 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 22:51:10.838444 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":496,"threads_started":1}
I20260809 22:51:10.839835 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.820136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:37809 (local address 127.24.136.3:53589)
0809 22:51:10.821031 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:10.821079 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:10.823120 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:10.823884 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:10.823894 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:10.824460 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:10.825140 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.825172 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.826445 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.826450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.827859 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.827874 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.828726 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.828732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.828963 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.829662 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:10.829691 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:10.832049 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:10.835736 (+ 3687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:10.835744 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:10.835766 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:10.836229 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:10.836699 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:10.836703 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:10.836706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:10.836924 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:10.837488 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:10.837495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:10.837870 (+ 375us) client_negotiation.cc:769] Sending connection context
0809 22:51:10.838112 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 22:51:10.838534 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":350,"threads_started":1}
I20260809 22:51:10.841293 27164 heartbeater.cc:347] Connected to a master server at 127.24.136.62:37809
I20260809 22:51:10.841606 27164 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:10.842422 27164 heartbeater.cc:511] Master 127.24.136.62:37809 requested a full tablet report, sending...
I20260809 22:51:10.844098 26372 ts_manager.cc:194] Registered new tserver with Master: 014a331cee2c469783e009e5b6fcc98f (127.24.136.3:39123)
I20260809 22:51:10.844934 26372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.136.3@KRBTEST.COM'} at 127.24.136.3:53589
I20260809 22:51:10.847754 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:51:10.864872 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.851985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44688 (local address 127.24.136.62:37809)
0809 22:51:10.852162 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:10.852169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:10.852616 (+ 447us) server_negotiation.cc:407] Connection header received
0809 22:51:10.852810 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:10.852814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:10.852894 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:10.852977 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:10.854434 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.855158 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.856504 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.856921 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.857051 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.859552 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:10.859557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:10.859560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:10.859597 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:10.862310 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:10.862967 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:10.862970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:10.862973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:10.863101 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:10.863503 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:10.863509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:10.863513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:10.863890 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:10.864083 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:10.864368 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 22:51:10.864561 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:51:10.868973 26372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:51:10.875886 26372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:51:10.909303 27203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.888457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:38053 (local address 127.0.0.1:56942)
0809 22:51:10.898266 (+ 9809us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:10.898339 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:10.898576 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:10.901965 (+ 3389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:10.901972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:10.902006 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:10.902326 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.902338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.904139 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.904144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.905689 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.905712 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.908191 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.908197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.908439 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.908501 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 22:51:10.908663 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 22:51:10.908888 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9438,"thread_start_us":11745,"threads_started":1}
I20260809 22:51:10.909345 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.888690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56942 (local address 127.24.136.1:38053)
0809 22:51:10.901273 (+ 12583us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:10.901284 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:10.901319 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:51:10.901431 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:10.901440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:10.901720 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:10.901919 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:10.902553 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.903990 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.905976 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.907646 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.907783 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.908047 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:10.908746 (+ 699us) server_negotiation.cc:299] Negotiation successful
0809 22:51:10.908967 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12341,"thread_start_us":12158,"threads_started":1}
I20260809 22:51:10.911047 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.889229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55362 (local address 127.24.136.2:44303)
0809 22:51:10.901634 (+ 12405us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:10.901645 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:10.901681 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:51:10.901789 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:10.901799 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:10.902053 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:10.903426 (+ 1373us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:10.903847 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.905575 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.908288 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.909849 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.910021 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.910250 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:10.910550 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 22:51:10.910749 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12215,"thread_start_us":12080,"threads_started":1}
I20260809 22:51:10.911048 27197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.888644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:44303 (local address 127.0.0.1:55362)
0809 22:51:10.899801 (+ 11157us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:10.899824 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:10.899967 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:10.902651 (+ 2684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:10.902658 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:10.902690 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:10.903171 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.903184 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.905801 (+ 2617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.905806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.908071 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.908089 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.910092 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.910103 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.910332 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.910371 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:51:10.910484 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 22:51:10.910640 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11012,"thread_start_us":10851,"threads_started":1}
I20260809 22:51:10.914676 27196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.889889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:39123 (local address 127.0.0.1:55600)
0809 22:51:10.900493 (+ 10604us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:10.900512 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:10.900611 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:10.907305 (+ 6694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:10.907309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:10.907344 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:10.907839 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.907853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.909872 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.909877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.911962 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:10.911980 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.913865 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.913872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:10.914054 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.914083 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 22:51:10.914164 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 22:51:10.914319 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10478,"thread_start_us":7812,"threads_started":1}
I20260809 22:51:10.914747 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:10.890540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55600 (local address 127.24.136.3:39123)
0809 22:51:10.906309 (+ 15769us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:10.906316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:10.906352 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:51:10.906441 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:10.906447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:10.906710 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:10.906965 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:10.908117 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.909704 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.912154 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:10.913609 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:10.913756 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:10.914011 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:10.914260 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 22:51:10.914433 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15575,"thread_start_us":15412,"threads_started":1}
I20260809 22:51:10.914502 26582 tablet_service.cc:1467] Processing CreateTablet for tablet eb6418bc269e45699502771300b633dc (DEFAULT_TABLE table=test-table [id=030630c6981f4a0a95b813036d75bb82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:10.915558 26825 tablet_service.cc:1467] Processing CreateTablet for tablet eb6418bc269e45699502771300b633dc (DEFAULT_TABLE table=test-table [id=030630c6981f4a0a95b813036d75bb82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:10.916469 26582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb6418bc269e45699502771300b633dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:10.917059 26825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb6418bc269e45699502771300b633dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:10.918422 27046 tablet_service.cc:1467] Processing CreateTablet for tablet eb6418bc269e45699502771300b633dc (DEFAULT_TABLE table=test-table [id=030630c6981f4a0a95b813036d75bb82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:10.920540 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb6418bc269e45699502771300b633dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:10.932350 27212 tablet_bootstrap.cc:492] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: Bootstrap starting.
I20260809 22:51:10.934957 27214 tablet_bootstrap.cc:492] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: Bootstrap starting.
I20260809 22:51:10.936650 27211 tablet_bootstrap.cc:492] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: Bootstrap starting.
I20260809 22:51:10.938349 27212 tablet_bootstrap.cc:654] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:10.938798 27214 tablet_bootstrap.cc:654] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:10.939839 27214 log.cc:824] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:10.939814 27212 log.cc:824] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:10.940286 27211 tablet_bootstrap.cc:654] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:10.941568 27211 log.cc:824] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:10.943049 27214 tablet_bootstrap.cc:492] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: No bootstrap required, opened a new log
I20260809 22:51:10.943128 27212 tablet_bootstrap.cc:492] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: No bootstrap required, opened a new log
I20260809 22:51:10.943514 27212 ts_tablet_manager.cc:1397] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 22:51:10.943547 27214 ts_tablet_manager.cc:1397] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260809 22:51:10.944787 27211 tablet_bootstrap.cc:492] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: No bootstrap required, opened a new log
I20260809 22:51:10.945072 27211 ts_tablet_manager.cc:1397] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:51:10.951746 27212 raft_consensus.cc:348] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } }
I20260809 22:51:10.952170 27212 raft_consensus.cc:374] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:10.952288 27212 raft_consensus.cc:729] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c74d9d61e624029a74085eef195abca, State: Initialized, Role: FOLLOWER
I20260809 22:51:10.952936 27212 consensus_queue.cc:260] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } }
I20260809 22:51:10.954077 26941 heartbeater.cc:503] Master 127.24.136.62:37809 was elected leader, sending a full tablet report...
I20260809 22:51:10.954455 27214 raft_consensus.cc:348] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } }
I20260809 22:51:10.954826 27212 ts_tablet_manager.cc:1428] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: Time spent starting tablet: real 0.011s user 0.005s sys 0.005s
I20260809 22:51:10.954907 27214 raft_consensus.cc:374] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:10.955003 27214 raft_consensus.cc:729] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 014a331cee2c469783e009e5b6fcc98f, State: Initialized, Role: FOLLOWER
I20260809 22:51:10.955670 27214 consensus_queue.cc:260] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } }
I20260809 22:51:10.955569 27211 raft_consensus.cc:348] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } }
I20260809 22:51:10.956000 27211 raft_consensus.cc:374] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:10.956107 27211 raft_consensus.cc:729] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d84ffbe6d2425a8e5fb27a974f10bf, State: Initialized, Role: FOLLOWER
I20260809 22:51:10.956828 27164 heartbeater.cc:503] Master 127.24.136.62:37809 was elected leader, sending a full tablet report...
I20260809 22:51:10.957489 27214 ts_tablet_manager.cc:1428] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260809 22:51:10.958048 27211 consensus_queue.cc:260] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } }
I20260809 22:51:10.959911 27211 ts_tablet_manager.cc:1428] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
W20260809 22:51:10.959975 26709 tablet.cc:2369] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:51:11.012691 26942 tablet.cc:2369] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:51:11.068370 27167 tablet.cc:2369] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:51:11.090330 27220 raft_consensus.cc:482] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:51:11.090632 27220 raft_consensus.cc:504] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } }
I20260809 22:51:11.092525 27220 leader_election.cc:288] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c74d9d61e624029a74085eef195abca (127.24.136.2:44303), 014a331cee2c469783e009e5b6fcc98f (127.24.136.3:39123)
I20260809 22:51:11.099327 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:11.093087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:39185 (local address 127.24.136.2:44303)
0809 22:51:11.093318 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:11.093324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:11.094178 (+ 854us) server_negotiation.cc:407] Connection header received
0809 22:51:11.094348 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:11.094352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:11.094414 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:11.094489 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:11.095050 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.095804 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.097620 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.098406 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.098555 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:11.098634 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:11.098932 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 22:51:11.099060 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:51:11.099491 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:11.093271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:57781 (local address 127.24.136.3:39123)
0809 22:51:11.093480 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:11.093486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:11.094276 (+ 790us) server_negotiation.cc:407] Connection header received
0809 22:51:11.094486 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:11.094491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:11.094553 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:11.094624 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:11.095229 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.096036 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.097796 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.098574 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.098729 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:11.098804 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:11.099080 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 22:51:11.099220 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:51:11.099574 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:11.093027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:44303 (local address 127.24.136.1:39185)
0809 22:51:11.094029 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:11.094050 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:11.094191 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:11.094525 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:11.094529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:11.094554 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:11.094908 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:11.094918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.095933 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.095937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:11.097458 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:11.097471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.098620 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.098627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:11.098783 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:11.098807 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:51:11.098892 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 22:51:11.099031 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"mutex_wait_us":25,"thread_start_us":822,"threads_started":1}
I20260809 22:51:11.099679 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:11.093080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:39123 (local address 127.24.136.1:57781)
0809 22:51:11.094153 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:11.094173 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:11.094336 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:11.094689 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:11.094693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:11.094721 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:11.095071 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:11.095080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.096181 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.096185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:11.097645 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:11.097657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.098800 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.098806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:11.098934 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:11.098960 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:51:11.099040 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 22:51:11.099165 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":3,"spinlock_wait_cycles":21888,"thread_start_us":495,"threads_started":1}
I20260809 22:51:11.100667 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb6418bc269e45699502771300b633dc" candidate_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c74d9d61e624029a74085eef195abca" is_pre_election: true
I20260809 22:51:11.100678 27084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb6418bc269e45699502771300b633dc" candidate_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "014a331cee2c469783e009e5b6fcc98f" is_pre_election: true
I20260809 22:51:11.101157 26859 raft_consensus.cc:2454] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d84ffbe6d2425a8e5fb27a974f10bf in term 0.
I20260809 22:51:11.101151 27084 raft_consensus.cc:2454] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d84ffbe6d2425a8e5fb27a974f10bf in term 0.
I20260809 22:51:11.101795 26505 leader_election.cc:302] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 014a331cee2c469783e009e5b6fcc98f, 36d84ffbe6d2425a8e5fb27a974f10bf; no voters:
I20260809 22:51:11.102188 27220 raft_consensus.cc:2781] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:51:11.102340 27220 raft_consensus.cc:482] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:51:11.102442 27220 raft_consensus.cc:3037] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:11.104719 27220 raft_consensus.cc:504] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } }
I20260809 22:51:11.105378 27220 leader_election.cc:288] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [CANDIDATE]: Term 1 election: Requested vote from peers 9c74d9d61e624029a74085eef195abca (127.24.136.2:44303), 014a331cee2c469783e009e5b6fcc98f (127.24.136.3:39123)
I20260809 22:51:11.105854 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb6418bc269e45699502771300b633dc" candidate_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c74d9d61e624029a74085eef195abca"
I20260809 22:51:11.105962 27084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb6418bc269e45699502771300b633dc" candidate_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "014a331cee2c469783e009e5b6fcc98f"
I20260809 22:51:11.106091 26859 raft_consensus.cc:3037] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:11.106164 27084 raft_consensus.cc:3037] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:11.109586 27084 raft_consensus.cc:2454] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d84ffbe6d2425a8e5fb27a974f10bf in term 1.
I20260809 22:51:11.109586 26859 raft_consensus.cc:2454] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d84ffbe6d2425a8e5fb27a974f10bf in term 1.
I20260809 22:51:11.110128 26505 leader_election.cc:302] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 014a331cee2c469783e009e5b6fcc98f, 36d84ffbe6d2425a8e5fb27a974f10bf, 9c74d9d61e624029a74085eef195abca; no voters:
I20260809 22:51:11.110522 27220 raft_consensus.cc:2781] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:11.111006 27220 raft_consensus.cc:686] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 1 LEADER]: Becoming Leader. State: Replica: 36d84ffbe6d2425a8e5fb27a974f10bf, State: Running, Role: LEADER
I20260809 22:51:11.111544 27220 consensus_queue.cc:237] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } }
I20260809 22:51:11.116220 26369 catalog_manager.cc:5495] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 36d84ffbe6d2425a8e5fb27a974f10bf (127.24.136.1). New cstate: current_term: 1 leader_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36d84ffbe6d2425a8e5fb27a974f10bf" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 38053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 } health_report { overall_health: UNKNOWN } } }
I20260809 22:51:11.148362 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:11.142759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56944 (local address 127.24.136.1:38053)
0809 22:51:11.142999 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:11.143004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:11.143022 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:51:11.143211 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:11.143214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:11.143296 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:11.143424 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:11.143869 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.144641 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.146058 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.146476 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.146576 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:11.147025 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:11.147699 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:11.147836 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:11.147987 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 22:51:11.148069 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:51:11.157629 27084 raft_consensus.cc:1261] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [term 1 FOLLOWER]: Refusing update from remote peer 36d84ffbe6d2425a8e5fb27a974f10bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:11.157634 26859 raft_consensus.cc:1261] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [term 1 FOLLOWER]: Refusing update from remote peer 36d84ffbe6d2425a8e5fb27a974f10bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:11.158663 27223 consensus_queue.cc:1035] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c74d9d61e624029a74085eef195abca" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 44303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:51:11.159116 27220 consensus_queue.cc:1035] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "014a331cee2c469783e009e5b6fcc98f" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 39123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:51:11.188985 26369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:51:11.189267 26369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:51:11.193104 26369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63e790c3f0ef42129b38c0f47699d5e4: Sending DeleteTablet for 3 replicas of tablet eb6418bc269e45699502771300b633dc
I20260809 22:51:11.194558 26582 tablet_service.cc:1514] Processing DeleteTablet for tablet eb6418bc269e45699502771300b633dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:11 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:56942
I20260809 22:51:11.194592 27046 tablet_service.cc:1514] Processing DeleteTablet for tablet eb6418bc269e45699502771300b633dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:11 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:55600
I20260809 22:51:11.195359 26825 tablet_service.cc:1514] Processing DeleteTablet for tablet eb6418bc269e45699502771300b633dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:11 UTC) from {username='kudu', principal='kudu/127.24.136.62@KRBTEST.COM'} at 127.0.0.1:55362
I20260809 22:51:11.196094 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 26475
I20260809 22:51:11.200431 27251 tablet_replica.cc:330] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: stopping tablet replica
I20260809 22:51:11.201661 27251 raft_consensus.cc:2229] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:51:11.202378 27251 raft_consensus.cc:2258] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:11.205112 27248 tablet_replica.cc:330] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: stopping tablet replica
I20260809 22:51:11.205687 27248 raft_consensus.cc:2229] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:51:11.206329 27248 raft_consensus.cc:2258] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:11.206974 27250 tablet_replica.cc:330] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: stopping tablet replica
I20260809 22:51:11.207451 27250 raft_consensus.cc:2229] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:51:11.208190 27250 raft_consensus.cc:2258] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:11.210085 27251 ts_tablet_manager.cc:1905] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:11.210729 27248 ts_tablet_manager.cc:1905] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:11.211788 27250 ts_tablet_manager.cc:1905] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:11.217278 27251 ts_tablet_manager.cc:1918] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 09 22:51:11 dist-test-slave-pww0 krb5kdc[26288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.1@KRBTEST.COM
I20260809 22:51:11.217608 27251 log.cc:1196] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/eb6418bc269e45699502771300b633dc
I20260809 22:51:11.217980 27248 ts_tablet_manager.cc:1918] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:51:11.218199 27248 log.cc:1196] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/eb6418bc269e45699502771300b633dc
I20260809 22:51:11.218458 27251 ts_tablet_manager.cc:1939] T eb6418bc269e45699502771300b633dc P 9c74d9d61e624029a74085eef195abca: Deleting consensus metadata
I20260809 22:51:11.218775 27248 ts_tablet_manager.cc:1939] T eb6418bc269e45699502771300b633dc P 014a331cee2c469783e009e5b6fcc98f: Deleting consensus metadata
I20260809 22:51:11.220677 26346 catalog_manager.cc:4841] TS 9c74d9d61e624029a74085eef195abca (127.24.136.2:44303): tablet eb6418bc269e45699502771300b633dc (table test-table [id=030630c6981f4a0a95b813036d75bb82]) successfully deleted
I20260809 22:51:11.221133 26341 catalog_manager.cc:4841] TS 014a331cee2c469783e009e5b6fcc98f (127.24.136.3:39123): tablet eb6418bc269e45699502771300b633dc (table test-table [id=030630c6981f4a0a95b813036d75bb82]) successfully deleted
I20260809 22:51:11.226187 27250 ts_tablet_manager.cc:1918] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:51:11.226464 27250 log.cc:1196] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315533505708-25120-0/minicluster-data/ts-0/wal/wals/eb6418bc269e45699502771300b633dc
I20260809 22:51:11.226922 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:11.197043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56948 (local address 127.24.136.1:38053)
0809 22:51:11.197247 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:11.197254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:11.201021 (+ 3767us) server_negotiation.cc:407] Connection header received
0809 22:51:11.201222 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:11.201228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:11.201346 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:11.201462 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:11.205106 (+ 3644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.206533 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.208179 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.208887 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.209016 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:11.218888 (+ 9872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:11.218909 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:11.218928 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:11.218976 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:11.223105 (+ 4129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:11.224210 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:11.224222 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:11.224226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:11.224452 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:11.224927 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:11.224933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:11.224937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:11.225529 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:11.225858 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:11.226235 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 22:51:11.226525 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 22:51:11.227121 27250 ts_tablet_manager.cc:1939] T eb6418bc269e45699502771300b633dc P 36d84ffbe6d2425a8e5fb27a974f10bf: Deleting consensus metadata
I20260809 22:51:11.227838 26698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:11.344449 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 26475
I20260809 22:51:11.344647 26343 catalog_manager.cc:4841] TS 36d84ffbe6d2425a8e5fb27a974f10bf (127.24.136.1:38053): tablet eb6418bc269e45699502771300b633dc (table test-table [id=030630c6981f4a0a95b813036d75bb82]) successfully deleted
I20260809 22:51:11.378118 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 26717
Aug 09 22:51:11 dist-test-slave-pww0 krb5kdc[26288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.2@KRBTEST.COM
I20260809 22:51:11.393836 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:11.378996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55370 (local address 127.24.136.2:44303)
0809 22:51:11.379193 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:11.379198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:11.379219 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:51:11.379357 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:11.379361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:11.379431 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:11.379557 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:11.380886 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.381671 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.383002 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.383414 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.383611 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:11.388054 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:11.388074 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:11.388092 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:11.388140 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:11.391032 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:11.391967 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:11.391976 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:11.391979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:11.392130 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:11.392480 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:11.392484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:11.392486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:11.392897 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:11.393143 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:11.393377 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 22:51:11.393561 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:51:11.394611 26928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:11.484639 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 26717
I20260809 22:51:11.512755 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 26953
Aug 09 22:51:11 dist-test-slave-pww0 krb5kdc[26288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.3@KRBTEST.COM
I20260809 22:51:11.529529 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:11.513616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55616 (local address 127.24.136.3:39123)
0809 22:51:11.513799 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:11.513804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:11.513828 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:51:11.514009 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:11.514012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:11.514093 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:11.514175 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:11.515620 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.516353 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.517832 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:11.518240 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:11.518348 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:11.523449 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:11.523495 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:11.523517 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:11.523561 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:11.526592 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:11.527443 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:11.527453 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:11.527485 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:11.527664 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:11.528055 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:11.528058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:11.528060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:11.528439 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:11.528699 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:11.529014 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 22:51:11.529195 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:51:11.530268 27158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:11.619539 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 26953
I20260809 22:51:11.639178 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 26309
I20260809 22:51:11.640653 26421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:11.751643 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 26309
2026-08-09T22:51:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4839 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T22:51:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:11Z Disabled control of system clock
I20260809 22:51:11.852664 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:33727
--webserver_interface=127.24.136.62
--webserver_port=0
--builtin_ntp_servers=127.24.136.20:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:33727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:51:12.212877 27267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:12.213191 27267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:12.213284 27267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:12.220834 27267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:12.220963 27267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:12.221012 27267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:12.221071 27267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:12.221114 27267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:12.231020 27267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:33727
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:33727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:12.232442 27267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:12.234298 27267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:12.247272 27276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:12.247188 27279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:12.247220 27277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:12.247992 27267 server_base.cc:1034] running on GCE node
I20260809 22:51:12.248955 27267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:12.250638 27267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:12.251870 27267 hybrid_clock.cc:648] HybridClock initialized: now 1786315872251814 us; error 55 us; skew 500 ppm
I20260809 22:51:12.252285 27267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:51:12.254954 27267 webserver.cc:460] Webserver started at http://127.24.136.62:32811/ using document root <none> and password file <none>
I20260809 22:51:12.255823 27267 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:12.255939 27267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:12.256292 27267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:12.259559 27267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "e011bff4bedf429fb6edb9a3c9052556"
format_stamp: "Formatted at 2026-08-09 22:51:12 on dist-test-slave-pww0"
server_key: "11a079320d2a1d48fa2465c9012ebe2e"
server_key_iv: "717fb90ecf89fe2f4a8fdf7637e5e0aa"
server_key_version: "encryptionkey@0"
I20260809 22:51:12.260511 27267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "e011bff4bedf429fb6edb9a3c9052556"
format_stamp: "Formatted at 2026-08-09 22:51:12 on dist-test-slave-pww0"
server_key: "11a079320d2a1d48fa2465c9012ebe2e"
server_key_iv: "717fb90ecf89fe2f4a8fdf7637e5e0aa"
server_key_version: "encryptionkey@0"
I20260809 22:51:12.266690 27267 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 22:51:12.271209 27290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:12.272953 27267 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:51:12.273185 27267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "e011bff4bedf429fb6edb9a3c9052556"
format_stamp: "Formatted at 2026-08-09 22:51:12 on dist-test-slave-pww0"
server_key: "11a079320d2a1d48fa2465c9012ebe2e"
server_key_iv: "717fb90ecf89fe2f4a8fdf7637e5e0aa"
server_key_version: "encryptionkey@0"
I20260809 22:51:12.273442 27267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:12.309072 27267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:12.310070 27267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:12.310418 27267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:12.329103 27267 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:33727
I20260809 22:51:12.329139 27383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:33727 every 8 connection(s)
I20260809 22:51:12.330670 27267 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:12.334954 27387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:12.335223 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 27267
I20260809 22:51:12.335412 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:12.335908 25120 external_mini_cluster.cc:1428] Setting key 3b8a531827003762d00e4fe32b049404
I20260809 22:51:12.351305 27387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556: Bootstrap starting.
I20260809 22:51:12.353876 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:12.338015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.24.136.62:33727)
0809 22:51:12.339296 (+ 1281us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:12.339312 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:12.339382 (+ 70us) server_negotiation.cc:407] Connection header received
0809 22:51:12.342083 (+ 2701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:12.342124 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:12.342963 (+ 839us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:12.343640 (+ 677us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:12.344292 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:12.346219 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:12.347902 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:12.349128 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:12.349348 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:12.350629 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:12.350641 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:12.350664 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:12.351452 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:12.351597 (+ 145us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:12.351628 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:12.351787 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:12.352041 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 22:51:12.352444 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":421,"threads_started":1}
I20260809 22:51:12.355576 27387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:12.356748 27387 log.cc:824] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:12.360023 27387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556: No bootstrap required, opened a new log
I20260809 22:51:12.367794 27387 raft_consensus.cc:348] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e011bff4bedf429fb6edb9a3c9052556" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33727 } }
I20260809 22:51:12.368261 27387 raft_consensus.cc:374] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:12.368376 27387 raft_consensus.cc:729] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e011bff4bedf429fb6edb9a3c9052556, State: Initialized, Role: FOLLOWER
I20260809 22:51:12.369117 27387 consensus_queue.cc:260] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e011bff4bedf429fb6edb9a3c9052556" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33727 } }
I20260809 22:51:12.369429 27387 raft_consensus.cc:388] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:12.369601 27387 raft_consensus.cc:482] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:12.369766 27387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:12.372222 27387 raft_consensus.cc:504] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e011bff4bedf429fb6edb9a3c9052556" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33727 } }
I20260809 22:51:12.372854 27387 leader_election.cc:302] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e011bff4bedf429fb6edb9a3c9052556; no voters:
I20260809 22:51:12.373497 27387 leader_election.cc:288] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:12.373656 27396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:12.374377 27396 raft_consensus.cc:686] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [term 1 LEADER]: Becoming Leader. State: Replica: e011bff4bedf429fb6edb9a3c9052556, State: Running, Role: LEADER
I20260809 22:51:12.374928 27396 consensus_queue.cc:237] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e011bff4bedf429fb6edb9a3c9052556" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33727 } }
I20260809 22:51:12.375735 27387 sys_catalog.cc:564] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:12.378052 27399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e011bff4bedf429fb6edb9a3c9052556. Latest consensus state: current_term: 1 leader_uuid: "e011bff4bedf429fb6edb9a3c9052556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e011bff4bedf429fb6edb9a3c9052556" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33727 } } }
I20260809 22:51:12.378502 27399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:12.379328 27397 sys_catalog.cc:455] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e011bff4bedf429fb6edb9a3c9052556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e011bff4bedf429fb6edb9a3c9052556" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33727 } } }
I20260809 22:51:12.379739 27397 sys_catalog.cc:458] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:12.380856 27405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:12.387544 27405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:12.397888 27405 catalog_manager.cc:1347] Generated new cluster ID: 98f8489dfdac43218c26d4eb5d3b62e9
I20260809 22:51:12.398042 27405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:12.414688 27405 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:12.416476 27405 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:12.423712 27405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556: Generated new TSK 0
I20260809 22:51:12.424623 27405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:12.431908 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.1:0
--local_ip_for_outbound_sockets=127.24.136.1
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33727
--builtin_ntp_servers=127.24.136.20:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:51:12.797240 27429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:12.797573 27429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:12.797744 27429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:12.805217 27429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:12.805349 27429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:12.805492 27429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.1
I20260809 22:51:12.814963 27429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.136.1
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.1
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:12.816397 27429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:12.818231 27429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:12.831933 27441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:12.831997 27440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:12.831970 27443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:12.832639 27429 server_base.cc:1034] running on GCE node
I20260809 22:51:12.833312 27429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:12.834323 27429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:12.835547 27429 hybrid_clock.cc:648] HybridClock initialized: now 1786315872835484 us; error 54 us; skew 500 ppm
I20260809 22:51:12.835876 27429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:51:12.838222 27429 webserver.cc:460] Webserver started at http://127.24.136.1:34423/ using document root <none> and password file <none>
I20260809 22:51:12.838929 27429 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:12.839026 27429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:12.839357 27429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:12.842377 27429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/data/instance:
uuid: "f88cff14b113476d9998818060536d85"
format_stamp: "Formatted at 2026-08-09 22:51:12 on dist-test-slave-pww0"
server_key: "5950e4fea1172d81a0b157824f8a3f86"
server_key_iv: "1e045ce98ac13bc7c671cc9bdb980fe0"
server_key_version: "encryptionkey@0"
I20260809 22:51:12.843191 27429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance:
uuid: "f88cff14b113476d9998818060536d85"
format_stamp: "Formatted at 2026-08-09 22:51:12 on dist-test-slave-pww0"
server_key: "5950e4fea1172d81a0b157824f8a3f86"
server_key_iv: "1e045ce98ac13bc7c671cc9bdb980fe0"
server_key_version: "encryptionkey@0"
I20260809 22:51:12.848697 27429 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:51:12.852490 27454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:12.854254 27429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:51:12.854421 27429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/wal
uuid: "f88cff14b113476d9998818060536d85"
format_stamp: "Formatted at 2026-08-09 22:51:12 on dist-test-slave-pww0"
server_key: "5950e4fea1172d81a0b157824f8a3f86"
server_key_iv: "1e045ce98ac13bc7c671cc9bdb980fe0"
server_key_version: "encryptionkey@0"
I20260809 22:51:12.854650 27429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:12.889684 27429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:12.890651 27429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:12.890961 27429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:12.892032 27429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:12.893585 27429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:12.893673 27429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:12.893784 27429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:12.893855 27429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:12.924906 27429 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.1:41513
I20260809 22:51:12.924981 27657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.1:41513 every 8 connection(s)
I20260809 22:51:12.926276 27429 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/data/info.pb
I20260809 22:51:12.927594 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 27429
I20260809 22:51:12.927788 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-0/wal/instance
I20260809 22:51:12.928391 25120 external_mini_cluster.cc:1428] Setting key 737aced48b3d07ab8a9b7da865a015ac
I20260809 22:51:12.931788 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.2:0
--local_ip_for_outbound_sockets=127.24.136.2
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33727
--builtin_ntp_servers=127.24.136.20:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:51:12.939450 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:12.929310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.1:48391 (local address 127.24.136.62:33727)
0809 22:51:12.929739 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:12.929745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:12.931153 (+ 1408us) server_negotiation.cc:407] Connection header received
0809 22:51:12.933020 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:12.933027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:12.933108 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:12.933201 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:12.934328 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:12.935039 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:12.936675 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:12.937086 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:12.937200 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:12.938587 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:12.938592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:12.938596 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:12.938616 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:12.938657 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:12.938669 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:12.938746 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:12.938978 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 22:51:12.939088 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":206,"threads_started":1}
I20260809 22:51:12.940500 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:12.929909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33727 (local address 127.24.136.1:48391)
0809 22:51:12.931001 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:12.931040 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:12.932757 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:12.933468 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:12.933478 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:12.933570 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:12.934153 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:12.934171 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:12.935202 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:12.935206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:12.936528 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:12.936542 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:12.937323 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:12.937331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:12.937565 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:12.938249 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:51:12.938266 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:12.938309 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:51:12.938397 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:12.938760 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:12.938773 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:12.938781 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:51:12.938952 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 22:51:12.939380 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":325,"threads_started":1}
I20260809 22:51:12.942078 27659 heartbeater.cc:347] Connected to a master server at 127.24.136.62:33727
I20260809 22:51:12.942389 27659 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:12.943142 27659 heartbeater.cc:511] Master 127.24.136.62:33727 requested a full tablet report, sending...
I20260809 22:51:12.945536 27321 ts_manager.cc:194] Registered new tserver with Master: f88cff14b113476d9998818060536d85 (127.24.136.1:41513)
I20260809 22:51:12.947602 27321 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.1:48391
W20260809 22:51:13.289191 27665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:13.289506 27665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:13.289613 27665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:13.296973 27665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:13.297099 27665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:13.297209 27665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.2
I20260809 22:51:13.306630 27665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.136.2
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.2
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:13.308008 27665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:13.309907 27665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:13.323382 27681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:13.323446 27684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:13.323700 27687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:13.324445 27665 server_base.cc:1034] running on GCE node
I20260809 22:51:13.325101 27665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:13.326174 27665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:13.327430 27665 hybrid_clock.cc:648] HybridClock initialized: now 1786315873327361 us; error 51 us; skew 500 ppm
I20260809 22:51:13.327832 27665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:51:13.330526 27665 webserver.cc:460] Webserver started at http://127.24.136.2:38473/ using document root <none> and password file <none>
I20260809 22:51:13.331259 27665 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:13.331367 27665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:13.331733 27665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:13.334658 27665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/data/instance:
uuid: "64b6cb4a57ef4a628129fd9b8496b235"
format_stamp: "Formatted at 2026-08-09 22:51:13 on dist-test-slave-pww0"
server_key: "aedb4ade50c2540f5ca0c27ea060ce75"
server_key_iv: "7b1a8b8fb3862688c32a2b887ea395eb"
server_key_version: "encryptionkey@0"
I20260809 22:51:13.335518 27665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance:
uuid: "64b6cb4a57ef4a628129fd9b8496b235"
format_stamp: "Formatted at 2026-08-09 22:51:13 on dist-test-slave-pww0"
server_key: "aedb4ade50c2540f5ca0c27ea060ce75"
server_key_iv: "7b1a8b8fb3862688c32a2b887ea395eb"
server_key_version: "encryptionkey@0"
I20260809 22:51:13.341118 27665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:51:13.344983 27693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:13.346468 27665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:13.346691 27665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/wal
uuid: "64b6cb4a57ef4a628129fd9b8496b235"
format_stamp: "Formatted at 2026-08-09 22:51:13 on dist-test-slave-pww0"
server_key: "aedb4ade50c2540f5ca0c27ea060ce75"
server_key_iv: "7b1a8b8fb3862688c32a2b887ea395eb"
server_key_version: "encryptionkey@0"
I20260809 22:51:13.346923 27665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:13.373539 27665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:13.374487 27665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:13.374818 27665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:13.375955 27665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:13.377610 27665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:13.377709 27665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:13.377832 27665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:13.377903 27665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:13.409739 27665 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.2:39859
I20260809 22:51:13.409833 27902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.2:39859 every 8 connection(s)
I20260809 22:51:13.411114 27665 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/data/info.pb
I20260809 22:51:13.421392 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 27665
I20260809 22:51:13.421576 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/wal/instance
I20260809 22:51:13.422027 25120 external_mini_cluster.cc:1428] Setting key 84f160f47ae87e25768ae8548a4ae45f
I20260809 22:51:13.424340 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.413822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.2:32923 (local address 127.24.136.62:33727)
0809 22:51:13.414022 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:13.414028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:13.415424 (+ 1396us) server_negotiation.cc:407] Connection header received
0809 22:51:13.417308 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:13.417314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:13.417392 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:13.417489 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:13.418629 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.419388 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.421035 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.421460 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.421584 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.423240 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:13.423247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:13.423254 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:13.423282 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:13.423320 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:13.423335 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:13.423425 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:13.423759 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 22:51:13.423900 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:51:13.425256 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.136.3:0
--local_ip_for_outbound_sockets=127.24.136.3
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33727
--builtin_ntp_servers=127.24.136.20:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:51:13.425478 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.414300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33727 (local address 127.24.136.2:32923)
0809 22:51:13.415242 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:13.415281 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:13.416990 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:13.417689 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:13.417698 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:13.417801 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:13.418432 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.418452 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.419576 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.419580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.420882 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.420896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.421748 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.421756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.421979 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.422791 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:51:13.422795 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:13.422869 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:51:13.423001 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:13.423442 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:13.423483 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:13.423494 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 22:51:13.423717 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 22:51:13.424192 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":270,"threads_started":1}
I20260809 22:51:13.427186 27905 heartbeater.cc:347] Connected to a master server at 127.24.136.62:33727
I20260809 22:51:13.427608 27905 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:13.428488 27905 heartbeater.cc:511] Master 127.24.136.62:33727 requested a full tablet report, sending...
I20260809 22:51:13.430171 27321 ts_manager.cc:194] Registered new tserver with Master: 64b6cb4a57ef4a628129fd9b8496b235 (127.24.136.2:39859)
I20260809 22:51:13.431213 27321 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.2:32923
W20260809 22:51:13.780479 27909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:13.780822 27909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:13.780985 27909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:13.788717 27909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:13.788856 27909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:13.789011 27909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.136.3
I20260809 22:51:13.798647 27909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.136.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.136.3
--webserver_port=0
--tserver_master_addrs=127.24.136.62:33727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.136.3
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:13.800092 27909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:13.801982 27909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:13.815735 27920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:13.815698 27917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:13.815798 27918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:13.816735 27909 server_base.cc:1034] running on GCE node
I20260809 22:51:13.817392 27909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:13.818455 27909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:13.819648 27909 hybrid_clock.cc:648] HybridClock initialized: now 1786315873819589 us; error 57 us; skew 500 ppm
I20260809 22:51:13.820003 27909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:51:13.822539 27909 webserver.cc:460] Webserver started at http://127.24.136.3:35581/ using document root <none> and password file <none>
I20260809 22:51:13.823305 27909 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:13.823403 27909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:13.823769 27909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:13.826871 27909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/data/instance:
uuid: "7f422751ea914c5fa98ae07306af9936"
format_stamp: "Formatted at 2026-08-09 22:51:13 on dist-test-slave-pww0"
server_key: "68dc44991ac3e790cc35fba37ef95a6a"
server_key_iv: "7e9d21a484ff8f5f960ffe14fe9d96db"
server_key_version: "encryptionkey@0"
I20260809 22:51:13.827848 27909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance:
uuid: "7f422751ea914c5fa98ae07306af9936"
format_stamp: "Formatted at 2026-08-09 22:51:13 on dist-test-slave-pww0"
server_key: "68dc44991ac3e790cc35fba37ef95a6a"
server_key_iv: "7e9d21a484ff8f5f960ffe14fe9d96db"
server_key_version: "encryptionkey@0"
I20260809 22:51:13.833559 27909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:51:13.838617 27930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:13.840353 27909 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:51:13.840544 27909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/wal
uuid: "7f422751ea914c5fa98ae07306af9936"
format_stamp: "Formatted at 2026-08-09 22:51:13 on dist-test-slave-pww0"
server_key: "68dc44991ac3e790cc35fba37ef95a6a"
server_key_iv: "7e9d21a484ff8f5f960ffe14fe9d96db"
server_key_version: "encryptionkey@0"
I20260809 22:51:13.840864 27909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:13.856299 27909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:13.857137 27909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:13.857411 27909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:13.858371 27909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:51:13.859970 27909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:51:13.860060 27909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:13.860146 27909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:51:13.860205 27909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:13.892091 27909 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.3:46847
I20260809 22:51:13.892150 28144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.3:46847 every 8 connection(s)
I20260809 22:51:13.893460 27909 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/data/info.pb
I20260809 22:51:13.899364 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 27909
I20260809 22:51:13.899582 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/wal/instance
I20260809 22:51:13.900015 25120 external_mini_cluster.cc:1428] Setting key 42f66eb330e9cdbae61fd18954d37040
I20260809 22:51:13.906926 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.896258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:53389 (local address 127.24.136.62:33727)
0809 22:51:13.896523 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:13.896528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:13.898003 (+ 1475us) server_negotiation.cc:407] Connection header received
0809 22:51:13.899922 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:13.899929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:13.900018 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:13.900108 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:13.901568 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.902663 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.904447 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.904835 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.904933 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.906198 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:13.906203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:13.906206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:13.906225 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:13.906274 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:13.906287 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:13.906357 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:13.906566 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 22:51:13.906654 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 22:51:13.908008 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.896856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33727 (local address 127.24.136.3:53389)
0809 22:51:13.897833 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:13.897870 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:13.899612 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:13.900384 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:13.900394 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:13.900518 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:13.901386 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.901414 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.902870 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.902877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.904313 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.904326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.905014 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.905019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.905234 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.905888 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:51:13.905890 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:13.905934 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:51:13.906018 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:13.906380 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:13.906390 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:13.906397 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 22:51:13.906555 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 22:51:13.906903 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":317,"threads_started":1}
I20260809 22:51:13.909418 28145 heartbeater.cc:347] Connected to a master server at 127.24.136.62:33727
I20260809 22:51:13.909731 28145 heartbeater.cc:464] Registering TS with master...
I20260809 22:51:13.910565 28145 heartbeater.cc:511] Master 127.24.136.62:33727 requested a full tablet report, sending...
I20260809 22:51:13.912108 27321 ts_manager.cc:194] Registered new tserver with Master: 7f422751ea914c5fa98ae07306af9936 (127.24.136.3:46847)
I20260809 22:51:13.912945 27321 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.136.3:53389
I20260809 22:51:13.915597 25120 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:51:13.925697 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.919863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.24.136.62:33727)
0809 22:51:13.920119 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:13.920127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:13.920475 (+ 348us) server_negotiation.cc:407] Connection header received
0809 22:51:13.920646 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:13.920651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:13.920720 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:13.920799 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:13.921241 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.921942 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.923343 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.923808 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.923903 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.925038 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:13.925044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:13.925047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:13.925066 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:13.925098 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:13.925125 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:13.925193 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:13.925344 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 22:51:13.925437 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:51:13.930012 27321 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:51:13.937152 27321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:51:13.950850 27659 heartbeater.cc:503] Master 127.24.136.62:33727 was elected leader, sending a full tablet report...
I20260809 22:51:13.972641 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.951146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43128 (local address 127.24.136.1:41513)
0809 22:51:13.960113 (+ 8967us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:13.960124 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:13.963493 (+ 3369us) server_negotiation.cc:407] Connection header received
0809 22:51:13.964466 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:13.964477 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:13.964803 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:13.965058 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:13.965763 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.967324 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.969919 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.971390 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.971545 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.971862 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:13.972099 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 22:51:13.972272 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8736,"thread_start_us":8560,"threads_started":1}
I20260809 22:51:13.972681 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.950419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.0.0.1:43128)
0809 22:51:13.963310 (+ 12891us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:13.963335 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:13.964159 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:13.965087 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:13.965094 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:13.965135 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:13.965543 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.965560 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.967508 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.967513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.969649 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.969662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.971623 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.971630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.971837 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.971896 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 22:51:13.972025 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 22:51:13.972235 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12710,"mutex_wait_us":88,"thread_start_us":1465,"threads_started":1}
I20260809 22:51:13.973507 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.952851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:39859 (local address 127.0.0.1:41848)
0809 22:51:13.964102 (+ 11251us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:13.964122 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:13.964263 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:13.966273 (+ 2010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:13.966278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:13.966304 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:13.966746 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.966757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.968761 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.968767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.970908 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.970922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.972773 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.972779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.973038 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.973070 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:51:13.973123 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 22:51:13.973229 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11107,"thread_start_us":10089,"threads_started":1}
I20260809 22:51:13.973533 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.950204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.3:46847 (local address 127.0.0.1:35536)
0809 22:51:13.952299 (+ 2095us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:13.952431 (+ 132us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:13.959492 (+ 7061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:13.965942 (+ 6450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:13.965947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:13.965974 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:13.966380 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.966399 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.968233 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.968239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.970405 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:13.970426 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.972649 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.972657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:13.972873 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.972908 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:51:13.972999 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:51:13.973146 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1807,"mutex_wait_us":1,"thread_start_us":13577,"threads_started":1}
I20260809 22:51:13.973968 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.950386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.24.136.3:46847)
0809 22:51:13.964879 (+ 14493us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:13.964887 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:13.964915 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:51:13.964997 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:13.965005 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:13.965251 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:13.965459 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:13.966760 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.968095 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.970658 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.972079 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.972223 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.972496 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:13.973443 (+ 947us) server_negotiation.cc:299] Negotiation successful
0809 22:51:13.973615 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14333,"thread_start_us":14176,"threads_started":1}
I20260809 22:51:13.974944 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:13.953401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.24.136.2:39859)
0809 22:51:13.965695 (+ 12294us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:13.965704 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:13.965732 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:51:13.965812 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:13.965817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:13.966042 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:13.966957 (+ 915us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:13.967189 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.968607 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.971140 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:13.972541 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:13.974138 (+ 1597us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:13.974388 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:13.974461 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 22:51:13.974641 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12153,"thread_start_us":12051,"threads_started":1}
I20260809 22:51:13.977259 27540 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f (DEFAULT_TABLE table=test-table [id=6446e7e623d24b3989b45c1842999318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:13.977771 28021 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f (DEFAULT_TABLE table=test-table [id=6446e7e623d24b3989b45c1842999318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:13.978749 27540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:13.978418 27769 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f (DEFAULT_TABLE table=test-table [id=6446e7e623d24b3989b45c1842999318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:51:13.979281 28021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:13.979760 27769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:13.991084 28198 tablet_bootstrap.cc:492] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85: Bootstrap starting.
I20260809 22:51:13.994957 28198 tablet_bootstrap.cc:654] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:13.996276 28200 tablet_bootstrap.cc:492] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: Bootstrap starting.
I20260809 22:51:13.996325 28198 log.cc:824] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:13.997231 28199 tablet_bootstrap.cc:492] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: Bootstrap starting.
I20260809 22:51:14.000084 28198 tablet_bootstrap.cc:492] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85: No bootstrap required, opened a new log
I20260809 22:51:14.000382 28198 ts_tablet_manager.cc:1397] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 22:51:14.001173 28199 tablet_bootstrap.cc:654] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:14.001348 28200 tablet_bootstrap.cc:654] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:14.002553 28200 log.cc:824] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:14.002557 28199 log.cc:824] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:14.007251 28199 tablet_bootstrap.cc:492] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: No bootstrap required, opened a new log
I20260809 22:51:14.007639 28199 ts_tablet_manager.cc:1397] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260809 22:51:14.008062 28200 tablet_bootstrap.cc:492] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: No bootstrap required, opened a new log
I20260809 22:51:14.008376 28200 ts_tablet_manager.cc:1397] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260809 22:51:14.009056 28198 raft_consensus.cc:348] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } }
I20260809 22:51:14.009670 28198 raft_consensus.cc:374] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:14.009833 28198 raft_consensus.cc:729] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f88cff14b113476d9998818060536d85, State: Initialized, Role: FOLLOWER
I20260809 22:51:14.010560 28198 consensus_queue.cc:260] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } }
I20260809 22:51:14.012459 28198 ts_tablet_manager.cc:1428] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260809 22:51:14.018658 28199 raft_consensus.cc:348] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } }
I20260809 22:51:14.019184 28199 raft_consensus.cc:374] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:14.019300 28199 raft_consensus.cc:729] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64b6cb4a57ef4a628129fd9b8496b235, State: Initialized, Role: FOLLOWER
I20260809 22:51:14.019562 28200 raft_consensus.cc:348] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } }
I20260809 22:51:14.020026 28200 raft_consensus.cc:374] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:14.020148 28200 raft_consensus.cc:729] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f422751ea914c5fa98ae07306af9936, State: Initialized, Role: FOLLOWER
I20260809 22:51:14.020037 28199 consensus_queue.cc:260] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } }
I20260809 22:51:14.020771 28200 consensus_queue.cc:260] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } }
I20260809 22:51:14.021188 27905 heartbeater.cc:503] Master 127.24.136.62:33727 was elected leader, sending a full tablet report...
I20260809 22:51:14.022033 28199 ts_tablet_manager.cc:1428] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260809 22:51:14.024315 28145 heartbeater.cc:503] Master 127.24.136.62:33727 was elected leader, sending a full tablet report...
I20260809 22:51:14.024885 28200 ts_tablet_manager.cc:1428] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260809 22:51:14.078589 28210 raft_consensus.cc:482] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:51:14.078920 28210 raft_consensus.cc:504] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } }
I20260809 22:51:14.080745 28210 leader_election.cc:288] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f88cff14b113476d9998818060536d85 (127.24.136.1:41513), 64b6cb4a57ef4a628129fd9b8496b235 (127.24.136.2:39859)
I20260809 22:51:14.086700 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.081284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:58817 (local address 127.24.136.1:41513)
0809 22:51:14.081524 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:14.081530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:14.081554 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:51:14.081711 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:14.081717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:14.081845 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:14.081936 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:14.082536 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.083316 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.085040 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.085846 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.086001 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:14.086074 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:14.086338 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 22:51:14.086465 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:51:14.086742 28157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.081154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.24.136.3:58817)
0809 22:51:14.081362 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.081387 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:14.081528 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:14.082146 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:14.082151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:14.082181 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:14.082362 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:14.082374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.083482 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.083488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:14.084874 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:14.084886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.085999 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.086005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:14.086173 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:14.086197 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:51:14.086279 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 22:51:14.086425 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 22:51:14.087237 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.081961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.136.3:56171 (local address 127.24.136.2:39859)
0809 22:51:14.082162 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:14.082167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:14.082391 (+ 224us) server_negotiation.cc:407] Connection header received
0809 22:51:14.082566 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:14.082570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:14.082643 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:14.082725 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:51:14.083242 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.083997 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.085647 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.086382 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.086519 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:14.086588 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:14.086885 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 22:51:14.087009 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:51:14.087816 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.081590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.2:39859 (local address 127.24.136.3:56171)
0809 22:51:14.082288 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.082302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:14.082417 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:14.082760 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:14.082764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:14.082788 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:51:14.083086 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:14.083096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.084140 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.084144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:14.085467 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:14.085479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.086540 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.086546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:14.086709 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:14.086735 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:51:14.087413 (+ 678us) client_negotiation.cc:240] Negotiation successful
0809 22:51:14.087564 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":372,"threads_started":1}
I20260809 22:51:14.087936 27573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bf3c6b61249338e6ec8341c1f8c0f" candidate_uuid: "7f422751ea914c5fa98ae07306af9936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f88cff14b113476d9998818060536d85" is_pre_election: true
I20260809 22:51:14.088575 27573 raft_consensus.cc:2454] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f422751ea914c5fa98ae07306af9936 in term 0.
I20260809 22:51:14.088896 27807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bf3c6b61249338e6ec8341c1f8c0f" candidate_uuid: "7f422751ea914c5fa98ae07306af9936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64b6cb4a57ef4a628129fd9b8496b235" is_pre_election: true
I20260809 22:51:14.089286 27933 leader_election.cc:302] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f422751ea914c5fa98ae07306af9936, f88cff14b113476d9998818060536d85; no voters:
I20260809 22:51:14.089447 27807 raft_consensus.cc:2454] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f422751ea914c5fa98ae07306af9936 in term 0.
I20260809 22:51:14.089738 28210 raft_consensus.cc:2781] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:51:14.089905 28210 raft_consensus.cc:482] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:51:14.090024 28210 raft_consensus.cc:3037] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:14.092450 28210 raft_consensus.cc:504] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } }
I20260809 22:51:14.093058 28210 leader_election.cc:288] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [CANDIDATE]: Term 1 election: Requested vote from peers f88cff14b113476d9998818060536d85 (127.24.136.1:41513), 64b6cb4a57ef4a628129fd9b8496b235 (127.24.136.2:39859)
I20260809 22:51:14.093678 27573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bf3c6b61249338e6ec8341c1f8c0f" candidate_uuid: "7f422751ea914c5fa98ae07306af9936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f88cff14b113476d9998818060536d85"
I20260809 22:51:14.093719 27807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bf3c6b61249338e6ec8341c1f8c0f" candidate_uuid: "7f422751ea914c5fa98ae07306af9936" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64b6cb4a57ef4a628129fd9b8496b235"
I20260809 22:51:14.093941 27573 raft_consensus.cc:3037] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:14.093941 27807 raft_consensus.cc:3037] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:14.097289 27573 raft_consensus.cc:2454] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f422751ea914c5fa98ae07306af9936 in term 1.
I20260809 22:51:14.097291 27807 raft_consensus.cc:2454] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f422751ea914c5fa98ae07306af9936 in term 1.
I20260809 22:51:14.097770 27933 leader_election.cc:302] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64b6cb4a57ef4a628129fd9b8496b235, 7f422751ea914c5fa98ae07306af9936; no voters:
I20260809 22:51:14.098052 28210 raft_consensus.cc:2781] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:14.098523 28210 raft_consensus.cc:686] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 1 LEADER]: Becoming Leader. State: Replica: 7f422751ea914c5fa98ae07306af9936, State: Running, Role: LEADER
I20260809 22:51:14.099021 28210 consensus_queue.cc:237] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } }
I20260809 22:51:14.104067 27320 catalog_manager.cc:5495] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f422751ea914c5fa98ae07306af9936 (127.24.136.3). New cstate: current_term: 1 leader_uuid: "7f422751ea914c5fa98ae07306af9936" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f422751ea914c5fa98ae07306af9936" member_type: VOTER last_known_addr { host: "127.24.136.3" port: 46847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 } health_report { overall_health: UNKNOWN } } }
I20260809 22:51:14.132747 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.127210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.24.136.3:46847)
0809 22:51:14.127441 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:14.127445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:14.127502 (+ 57us) server_negotiation.cc:407] Connection header received
0809 22:51:14.127637 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:14.127641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:14.127729 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:14.127909 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:14.128220 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.128997 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.130380 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.130819 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.130920 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:14.131514 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:14.132159 (+ 645us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:14.132300 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:14.132355 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 22:51:14.132437 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 22:51:14.141897 27573 raft_consensus.cc:1261] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [term 1 FOLLOWER]: Refusing update from remote peer 7f422751ea914c5fa98ae07306af9936: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:14.141903 27807 raft_consensus.cc:1261] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [term 1 FOLLOWER]: Refusing update from remote peer 7f422751ea914c5fa98ae07306af9936: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:14.143019 28213 consensus_queue.cc:1035] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b6cb4a57ef4a628129fd9b8496b235" member_type: VOTER last_known_addr { host: "127.24.136.2" port: 39859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:51:14.143440 28210 consensus_queue.cc:1035] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f88cff14b113476d9998818060536d85" member_type: VOTER last_known_addr { host: "127.24.136.1" port: 41513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:51:14.145293 28150 tablet.cc:2369] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:51:14.173015 27321 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:51:14.173331 27321 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:51:14.177383 27321 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e011bff4bedf429fb6edb9a3c9052556: Sending DeleteTablet for 3 replicas of tablet 0a2bf3c6b61249338e6ec8341c1f8c0f
I20260809 22:51:14.178865 27769 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:14 UTC) from {username='slave'} at 127.0.0.1:41848
I20260809 22:51:14.178823 28021 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:14 UTC) from {username='slave'} at 127.0.0.1:35536
I20260809 22:51:14.179013 27540 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:51:14 UTC) from {username='slave'} at 127.0.0.1:43128
I20260809 22:51:14.180614 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27429
I20260809 22:51:14.190450 28239 tablet_replica.cc:330] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: stopping tablet replica
I20260809 22:51:14.191006 28239 raft_consensus.cc:2229] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:51:14.193157 28239 raft_consensus.cc:2258] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:14.194056 28244 tablet_replica.cc:330] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: stopping tablet replica
I20260809 22:51:14.195179 28244 raft_consensus.cc:2229] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:51:14.196393 28244 raft_consensus.cc:2258] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:14.197938 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.182216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43138 (local address 127.24.136.1:41513)
0809 22:51:14.182401 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:14.182407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:14.182428 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:51:14.182733 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:14.182739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:14.182868 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:14.182956 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:14.183939 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.185077 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.193028 (+ 7951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.193854 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.193990 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:14.195884 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:14.195898 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:14.195917 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:14.196799 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:14.196880 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:14.196913 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:14.197031 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:14.197351 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 22:51:14.197507 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:51:14.198089 28239 ts_tablet_manager.cc:1905] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:14.199635 27647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:14.206169 28244 ts_tablet_manager.cc:1905] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:51:14.214460 28239 ts_tablet_manager.cc:1918] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:51:14.215544 28239 log.cc:1196] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-1/wal/wals/0a2bf3c6b61249338e6ec8341c1f8c0f
I20260809 22:51:14.219309 28244 ts_tablet_manager.cc:1918] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:51:14.220877 28239 ts_tablet_manager.cc:1939] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 64b6cb4a57ef4a628129fd9b8496b235: Deleting consensus metadata
I20260809 22:51:14.223560 28244 log.cc:1196] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: Deleting WAL directory at /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315533505708-25120-0/minicluster-data/ts-2/wal/wals/0a2bf3c6b61249338e6ec8341c1f8c0f
I20260809 22:51:14.223707 27294 catalog_manager.cc:4841] TS 64b6cb4a57ef4a628129fd9b8496b235 (127.24.136.2:39859): tablet 0a2bf3c6b61249338e6ec8341c1f8c0f (table test-table [id=6446e7e623d24b3989b45c1842999318]) successfully deleted
I20260809 22:51:14.224275 28244 ts_tablet_manager.cc:1939] T 0a2bf3c6b61249338e6ec8341c1f8c0f P 7f422751ea914c5fa98ae07306af9936: Deleting consensus metadata
I20260809 22:51:14.225916 27298 catalog_manager.cc:4841] TS 7f422751ea914c5fa98ae07306af9936 (127.24.136.3:46847): tablet 0a2bf3c6b61249338e6ec8341c1f8c0f (table test-table [id=6446e7e623d24b3989b45c1842999318]) successfully deleted
I20260809 22:51:14.319015 28241 tablet_replica.cc:330] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85: stopping tablet replica
I20260809 22:51:14.319521 28241 raft_consensus.cc:2229] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:51:14.320071 28241 raft_consensus.cc:2258] T 0a2bf3c6b61249338e6ec8341c1f8c0f P f88cff14b113476d9998818060536d85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:51:14.321081 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 27429
I20260809 22:51:14.343205 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27665
W20260809 22:51:14.343144 27294 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.136.1:41513 (error 108)
I20260809 22:51:14.345484 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.344966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.0.0.1:43152)
0809 22:51:14.345176 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.345307 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 22:51:14.345955 27294 catalog_manager.cc:4584] TS f88cff14b113476d9998818060536d85 (127.24.136.1:41513): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a2bf3c6b61249338e6ec8341c1f8c0f: Network error: Client connection negotiation failed: client connection to 127.24.136.1:41513: connect: Connection refused (error 111)
I20260809 22:51:14.353381 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.344484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.24.136.2:39859)
0809 22:51:14.344672 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:14.344679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:14.344700 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:51:14.345192 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:14.345199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:14.345353 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:14.345454 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:14.346166 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.347329 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.349317 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.349974 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.350116 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:14.351412 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:14.351426 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:14.351449 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:14.352518 (+ 1069us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:14.352588 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:14.352618 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:14.352729 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:14.352883 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 22:51:14.353008 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:51:14.354110 27894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:14.400063 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.399516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.0.0.1:43168)
0809 22:51:14.399747 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.399861 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 22:51:14.401391 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.400916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.0.0.1:43176)
0809 22:51:14.401086 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.401240 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 22:51:14.461982 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.461475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.0.0.1:43182)
0809 22:51:14.461704 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.461798 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 22:51:14.463119 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.462750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.0.0.1:43190)
0809 22:51:14.462908 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.462984 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 22:51:14.503867 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 27665
I20260809 22:51:14.528362 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27909
I20260809 22:51:14.535919 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.529208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.24.136.3:46847)
0809 22:51:14.529429 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:14.529435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:14.529457 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:51:14.529613 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:14.529619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:14.529747 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:14.529837 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:14.530310 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.531069 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.532445 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:14.532927 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:14.533049 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:14.534363 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:14.534373 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:14.534392 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:14.535028 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:14.535083 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:14.535111 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:14.535226 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:14.535404 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 22:51:14.535558 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 22:51:14.536788 28137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:14.543812 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.543172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.0.0.1:43196)
0809 22:51:14.543443 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.543616 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 22:51:14.545063 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.544600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.0.0.1:43200)
0809 22:51:14.544819 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.544914 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260809 22:51:14.660856 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 27909
I20260809 22:51:14.689855 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 27267
I20260809 22:51:14.691123 27379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:14.806588 28184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:14.806046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.1:41513 (local address 127.0.0.1:43210)
0809 22:51:14.806263 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:14.806355 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.1:41513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 22:51:14.806777 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 27267
2026-08-09T22:51:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3053 ms)
[----------] 6 tests from EncryptionPolicyTest (23837 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:14 dist-test-slave-pww0 krb5kdc[28255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:14 dist-test-slave-pww0 krb5kdc[28255](info): set up 2 sockets
Aug 09 22:51:14 dist-test-slave-pww0 krb5kdc[28255](info): commencing operation
krb5kdc: starting...
W20260809 22:51:16.101297 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:16 dist-test-slave-pww0 krb5kdc[28255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:16.431689 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:16.790934 28274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:16.791275 28274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:16.791373 28274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:16.798920 28274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:16.799053 28274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:16.799108 28274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:16.799172 28274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:16.799216 28274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:16.809628 28274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46815
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:16.811008 28274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:16.812376 28274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:16.824539 28281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:16.824510 28284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:16.824466 28282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:16.825587 28274 server_base.cc:1034] running on GCE node
I20260809 22:51:16.826470 28274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:16.828086 28274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:16.829339 28274 hybrid_clock.cc:648] HybridClock initialized: now 1786315876829275 us; error 68 us; skew 500 ppm
I20260809 22:51:16.829731 28274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:16 dist-test-slave-pww0 krb5kdc[28255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:16.837689 28274 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:16.839419 28274 webserver.cc:460] Webserver started at http://127.0.0.1:37547/ using document root <none> and password file <none>
I20260809 22:51:16.840204 28274 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:16.840342 28274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:16.840669 28274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:16.844022 28274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "cf42955aab1e46af961367b36d1f6296"
format_stamp: "Formatted at 2026-08-09 22:51:16 on dist-test-slave-pww0"
server_key: "5f8be2844bf7b6efb45ecd0952e75531"
server_key_iv: "1f6b592edb0bb728a562454d0915b916"
server_key_version: "encryptionkey@0"
I20260809 22:51:16.844847 28274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "cf42955aab1e46af961367b36d1f6296"
format_stamp: "Formatted at 2026-08-09 22:51:16 on dist-test-slave-pww0"
server_key: "5f8be2844bf7b6efb45ecd0952e75531"
server_key_iv: "1f6b592edb0bb728a562454d0915b916"
server_key_version: "encryptionkey@0"
I20260809 22:51:16.850790 28274 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 22:51:16.854846 28302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:16.856398 28274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:51:16.856597 28274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "cf42955aab1e46af961367b36d1f6296"
format_stamp: "Formatted at 2026-08-09 22:51:16 on dist-test-slave-pww0"
server_key: "5f8be2844bf7b6efb45ecd0952e75531"
server_key_iv: "1f6b592edb0bb728a562454d0915b916"
server_key_version: "encryptionkey@0"
I20260809 22:51:16.856808 28274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:16.871860 28274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:16.876063 28274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:16.876344 28274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:16.894831 28274 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46815
I20260809 22:51:16.894913 28395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46815 every 8 connection(s)
I20260809 22:51:16.896324 28274 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:16.900274 28396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:16.905478 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 28274
I20260809 22:51:16.905702 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:16.906514 25120 external_mini_cluster.cc:1428] Setting key 75a1c8ae61dd9cc59e74e72378cd7f1b
I20260809 22:51:16.912587 28396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296: Bootstrap starting.
I20260809 22:51:16.915989 28396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:16.917356 28396 log.cc:824] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:16.921176 28396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296: No bootstrap required, opened a new log
Aug 09 22:51:16 dist-test-slave-pww0 krb5kdc[28255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:51:16.930488 28396 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42955aab1e46af961367b36d1f6296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46815 } }
I20260809 22:51:16.930891 28396 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:16.931008 28396 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf42955aab1e46af961367b36d1f6296, State: Initialized, Role: FOLLOWER
I20260809 22:51:16.931778 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:16.908785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55101 (local address 127.0.0.1:46815)
0809 22:51:16.909622 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:16.909638 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:16.909690 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:51:16.911572 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:16.911601 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:16.912031 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:16.912451 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:16.914049 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:16.915371 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:16.917334 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:16.918322 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:16.918559 (+ 237us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:16.923611 (+ 5052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:16.923621 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:16.923637 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:16.923682 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:16.926865 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:16.927948 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:16.927956 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:16.927963 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:16.928173 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:16.928668 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:16.928671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:16.928673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:16.929284 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:16.929538 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:16.930164 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 22:51:16.930591 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":340,"threads_started":1}
I20260809 22:51:16.931697 28396 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42955aab1e46af961367b36d1f6296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46815 } }
I20260809 22:51:16.931946 28396 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:16.932091 28396 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:16.932299 28396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:16.934799 28396 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42955aab1e46af961367b36d1f6296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46815 } }
I20260809 22:51:16.935362 28396 leader_election.cc:302] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf42955aab1e46af961367b36d1f6296; no voters:
I20260809 22:51:16.936020 28396 leader_election.cc:288] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:16.936177 28406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:16.936281 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:16.938056 28396 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:16.943806 28406 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [term 1 LEADER]: Becoming Leader. State: Replica: cf42955aab1e46af961367b36d1f6296, State: Running, Role: LEADER
I20260809 22:51:16.944465 28406 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42955aab1e46af961367b36d1f6296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46815 } }
I20260809 22:51:16.948393 28408 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf42955aab1e46af961367b36d1f6296. Latest consensus state: current_term: 1 leader_uuid: "cf42955aab1e46af961367b36d1f6296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42955aab1e46af961367b36d1f6296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46815 } } }
I20260809 22:51:16.949318 28408 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:16.948896 28407 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf42955aab1e46af961367b36d1f6296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf42955aab1e46af961367b36d1f6296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46815 } } }
I20260809 22:51:16.954105 28407 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296 [sys.catalog]: This master's current role is: LEADER
W20260809 22:51:16.962381 28430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:51:16.962628 28430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:51:16.962764 28431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:16.963541 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:16.944522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50515 (local address 127.0.0.1:46815)
0809 22:51:16.944743 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:16.944749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:16.945328 (+ 579us) server_negotiation.cc:407] Connection header received
0809 22:51:16.945573 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:16.945579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:16.945680 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:16.945762 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:16.948117 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:16.949970 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:16.952089 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:16.952848 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:16.952957 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:16.957106 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:16.957114 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:16.957118 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:16.957159 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:16.960198 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:16.961200 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:16.961204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:16.961207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:16.961337 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:16.961923 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:16.961926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:16.961928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:16.962299 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:16.962596 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:16.962986 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 22:51:16.963193 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:51:16.964663 28431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:16.972826 28431 catalog_manager.cc:1347] Generated new cluster ID: f96f71cbdd9043958b64e0e4ebff22c1
I20260809 22:51:16.972958 28431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:16.986500 28431 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:16.988437 28431 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:17.003064 28431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf42955aab1e46af961367b36d1f6296: Generated new TSK 0
I20260809 22:51:17.004077 28431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:17.016075 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28274
I20260809 22:51:17.018762 28388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:17.112666 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 28274
W20260809 22:51:17.146446 28252 connection.cc:535] client connection to 127.0.0.1:46815 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46815): Connection reset by peer (error 104)
2026-08-09T22:51:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:17 dist-test-slave-pww0 krb5kdc[28442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:17 dist-test-slave-pww0 krb5kdc[28442](info): set up 2 sockets
Aug 09 22:51:17 dist-test-slave-pww0 krb5kdc[28442](info): commencing operation
krb5kdc: starting...
W20260809 22:51:18.406895 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:18 dist-test-slave-pww0 krb5kdc[28442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:18.734784 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:19.102615 28462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:19.102955 28462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:19.103072 28462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:19.110702 28462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:19.110845 28462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:19.110900 28462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:19.110971 28462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:19.111022 28462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:19.122113 28462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:19.123565 28462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:19.125152 28462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:19.139276 28470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:19.139263 28469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:19.139837 28473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:19.140216 28462 server_base.cc:1034] running on GCE node
I20260809 22:51:19.141050 28462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:19.142618 28462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:19.143817 28462 hybrid_clock.cc:648] HybridClock initialized: now 1786315879143750 us; error 59 us; skew 500 ppm
I20260809 22:51:19.144174 28462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:19 dist-test-slave-pww0 krb5kdc[28442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315879, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:19.147620 28462 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:19.149284 28462 webserver.cc:460] Webserver started at http://127.0.0.1:36743/ using document root <none> and password file <none>
I20260809 22:51:19.150022 28462 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:19.150142 28462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:19.150470 28462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:19.153848 28462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "32b2b39f7c4d411e99bc822b7cee6518"
format_stamp: "Formatted at 2026-08-09 22:51:19 on dist-test-slave-pww0"
server_key: "9e68f34557ca271f2199ab2b832917ed"
server_key_iv: "a9e5cf81cb4f11fddca01dc52743b7fb"
server_key_version: "encryptionkey@0"
I20260809 22:51:19.154704 28462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "32b2b39f7c4d411e99bc822b7cee6518"
format_stamp: "Formatted at 2026-08-09 22:51:19 on dist-test-slave-pww0"
server_key: "9e68f34557ca271f2199ab2b832917ed"
server_key_iv: "a9e5cf81cb4f11fddca01dc52743b7fb"
server_key_version: "encryptionkey@0"
I20260809 22:51:19.161062 28462 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 22:51:19.165712 28482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:19.168860 28462 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260809 22:51:19.169076 28462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "32b2b39f7c4d411e99bc822b7cee6518"
format_stamp: "Formatted at 2026-08-09 22:51:19 on dist-test-slave-pww0"
server_key: "9e68f34557ca271f2199ab2b832917ed"
server_key_iv: "a9e5cf81cb4f11fddca01dc52743b7fb"
server_key_version: "encryptionkey@0"
I20260809 22:51:19.169328 28462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:19.208665 28462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:19.214110 28462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:19.214401 28462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:19.231740 28462 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43699
I20260809 22:51:19.231774 28570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43699 every 8 connection(s)
I20260809 22:51:19.233290 28462 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:19.237416 28572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:19.239717 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 28462
I20260809 22:51:19.239900 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:19.240351 25120 external_mini_cluster.cc:1428] Setting key b442d96f7de00d350bb38101a9033dc7
I20260809 22:51:19.251643 28572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518: Bootstrap starting.
I20260809 22:51:19.255781 28572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:19.256596 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:19.242321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50741 (local address 127.0.0.1:43699)
0809 22:51:19.243201 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:19.243217 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:19.243288 (+ 71us) server_negotiation.cc:407] Connection header received
0809 22:51:19.245702 (+ 2414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:19.245733 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:19.246303 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:19.246775 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:19.247355 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:19.249471 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:19.250966 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:19.252060 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:19.252269 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:19.253543 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:19.253554 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:19.253574 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:19.254321 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:19.254429 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:19.254464 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:19.254628 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:19.254946 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 22:51:19.255128 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":344,"threads_started":1}
I20260809 22:51:19.257201 28572 log.cc:824] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:19.260597 28572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518: No bootstrap required, opened a new log
I20260809 22:51:19.261075 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:19.269621 28572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b2b39f7c4d411e99bc822b7cee6518" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43699 } }
I20260809 22:51:19.270169 28572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:19.270306 28572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32b2b39f7c4d411e99bc822b7cee6518, State: Initialized, Role: FOLLOWER
I20260809 22:51:19.271003 28572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b2b39f7c4d411e99bc822b7cee6518" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43699 } }
I20260809 22:51:19.271243 28572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:19.271378 28572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:19.271625 28572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:19.272742 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:19.265492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58601 (local address 127.0.0.1:43699)
0809 22:51:19.265735 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:19.265742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:19.266284 (+ 542us) server_negotiation.cc:407] Connection header received
0809 22:51:19.266449 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:19.266454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:19.266542 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:19.266676 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:19.267160 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:19.268390 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:19.269799 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:19.270477 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:19.270604 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:19.271899 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:19.271905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:19.271908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:19.271929 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:19.271979 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:19.271998 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:19.272096 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:19.272320 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 22:51:19.272373 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:51:19.274644 28572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b2b39f7c4d411e99bc822b7cee6518" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43699 } }
I20260809 22:51:19.275247 28572 leader_election.cc:302] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32b2b39f7c4d411e99bc822b7cee6518; no voters:
I20260809 22:51:19.275930 28572 leader_election.cc:288] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:19.276103 28594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:19.277484 28594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [term 1 LEADER]: Becoming Leader. State: Replica: 32b2b39f7c4d411e99bc822b7cee6518, State: Running, Role: LEADER
I20260809 22:51:19.278268 28594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b2b39f7c4d411e99bc822b7cee6518" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43699 } }
I20260809 22:51:19.278429 28572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:19.281491 28599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32b2b39f7c4d411e99bc822b7cee6518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b2b39f7c4d411e99bc822b7cee6518" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43699 } } }
I20260809 22:51:19.281525 28600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32b2b39f7c4d411e99bc822b7cee6518. Latest consensus state: current_term: 1 leader_uuid: "32b2b39f7c4d411e99bc822b7cee6518" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b2b39f7c4d411e99bc822b7cee6518" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43699 } } }
I20260809 22:51:19.281997 28599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:19.281996 28600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:19.283732 28608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:19.288473 28608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:19.299978 28608 catalog_manager.cc:1347] Generated new cluster ID: a8c56a6543284672b84e9d0a178e99f8
I20260809 22:51:19.300120 28608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:19.309005 28608 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:19.310359 28608 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:19.320282 28608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32b2b39f7c4d411e99bc822b7cee6518: Generated new TSK 0
I20260809 22:51:19.321175 28608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:19.332706 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28462
W20260809 22:51:19.334115 28565 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50741
W20260809 22:51:19.335075 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28462: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:19.335265 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 28462
W20260809 22:51:19.361898 28438 connection.cc:535] client connection to 127.0.0.1:43699 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43699): Connection reset by peer (error 104)
2026-08-09T22:51:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2215 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:19 dist-test-slave-pww0 krb5kdc[28639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:19 dist-test-slave-pww0 krb5kdc[28639](info): set up 2 sockets
Aug 09 22:51:19 dist-test-slave-pww0 krb5kdc[28639](info): commencing operation
krb5kdc: starting...
W20260809 22:51:20.617267 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:20 dist-test-slave-pww0 krb5kdc[28639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:20.951009 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:21.323280 28656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:21.323699 28656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:21.323864 28656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:21.331743 28656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:21.331876 28656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:21.331931 28656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:21.331992 28656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:21.332036 28656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:21.342509 28656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33259
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:21.343966 28656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:21.345364 28656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:21.358067 28665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:21.358018 28668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:21.358067 28666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:21.359292 28656 server_base.cc:1034] running on GCE node
I20260809 22:51:21.360280 28656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:21.361928 28656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:21.363163 28656 hybrid_clock.cc:648] HybridClock initialized: now 1786315881363114 us; error 51 us; skew 500 ppm
I20260809 22:51:21.363613 28656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:21 dist-test-slave-pww0 krb5kdc[28639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:21.367072 28656 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:21.368880 28656 webserver.cc:460] Webserver started at http://127.0.0.1:40189/ using document root <none> and password file <none>
I20260809 22:51:21.369642 28656 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:21.369777 28656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:21.370105 28656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:21.373548 28656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "35a9983c8729422794a29f25edb413bf"
format_stamp: "Formatted at 2026-08-09 22:51:21 on dist-test-slave-pww0"
server_key: "d68533bc2c677a838597f987dab3038e"
server_key_iv: "04629491cc57bed365194c228424a495"
server_key_version: "encryptionkey@0"
I20260809 22:51:21.374425 28656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "35a9983c8729422794a29f25edb413bf"
format_stamp: "Formatted at 2026-08-09 22:51:21 on dist-test-slave-pww0"
server_key: "d68533bc2c677a838597f987dab3038e"
server_key_iv: "04629491cc57bed365194c228424a495"
server_key_version: "encryptionkey@0"
I20260809 22:51:21.380367 28656 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 22:51:21.384479 28678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:21.386087 28656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:21.386296 28656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "35a9983c8729422794a29f25edb413bf"
format_stamp: "Formatted at 2026-08-09 22:51:21 on dist-test-slave-pww0"
server_key: "d68533bc2c677a838597f987dab3038e"
server_key_iv: "04629491cc57bed365194c228424a495"
server_key_version: "encryptionkey@0"
I20260809 22:51:21.386511 28656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:21.392868 28656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:21.397333 28656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:21.397634 28656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:21.416756 28656 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33259
I20260809 22:51:21.416806 28771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33259 every 8 connection(s)
I20260809 22:51:21.418322 28656 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:21.422338 28776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:21.425050 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 28656
I20260809 22:51:21.425238 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:21.425782 25120 external_mini_cluster.cc:1428] Setting key fcaf1996064d50a9afbdd3adf09929a4
I20260809 22:51:21.434172 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:21.427752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45065 (local address 127.0.0.1:33259)
0809 22:51:21.428683 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:21.428698 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:21.428747 (+ 49us) server_negotiation.cc:407] Connection header received
0809 22:51:21.430509 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:21.430538 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:21.430938 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:21.431313 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:21.432471 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:21.432477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:21.432491 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:21.432509 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:21.432574 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:21.432594 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:21.432789 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:21.432899 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 22:51:21.433118 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":356,"threads_started":1}
I20260809 22:51:21.435549 28776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf: Bootstrap starting.
I20260809 22:51:21.438071 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:21.439157 28776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:21.440603 28776 log.cc:824] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:21.444052 28776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf: No bootstrap required, opened a new log
I20260809 22:51:21.445863 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:21.443401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55205 (local address 127.0.0.1:33259)
0809 22:51:21.443611 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:21.443618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:21.443926 (+ 308us) server_negotiation.cc:407] Connection header received
0809 22:51:21.444078 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:21.444083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:21.444147 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:21.444261 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:21.445225 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:21.445230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:21.445235 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:21.445254 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:21.445303 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:21.445325 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:21.445464 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:21.445510 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 22:51:21.445572 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 22:51:21.452471 28776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a9983c8729422794a29f25edb413bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260809 22:51:21.452876 28776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:21.452999 28776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a9983c8729422794a29f25edb413bf, State: Initialized, Role: FOLLOWER
I20260809 22:51:21.453631 28776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a9983c8729422794a29f25edb413bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260809 22:51:21.453946 28776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:21.454106 28776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:21.454299 28776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:21.456686 28776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a9983c8729422794a29f25edb413bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260809 22:51:21.457358 28776 leader_election.cc:302] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35a9983c8729422794a29f25edb413bf; no voters:
I20260809 22:51:21.458092 28776 leader_election.cc:288] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:21.458216 28789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:21.458882 28789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [term 1 LEADER]: Becoming Leader. State: Replica: 35a9983c8729422794a29f25edb413bf, State: Running, Role: LEADER
I20260809 22:51:21.459430 28789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a9983c8729422794a29f25edb413bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260809 22:51:21.460330 28776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:21.462313 28791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35a9983c8729422794a29f25edb413bf. Latest consensus state: current_term: 1 leader_uuid: "35a9983c8729422794a29f25edb413bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a9983c8729422794a29f25edb413bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } } }
I20260809 22:51:21.462819 28791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:21.463424 28790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35a9983c8729422794a29f25edb413bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a9983c8729422794a29f25edb413bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } } }
I20260809 22:51:21.463749 28790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:21.465291 28804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:21.470868 28804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:21.480710 28804 catalog_manager.cc:1347] Generated new cluster ID: 824da9e2edff437db2e0e6699bf8fa42
I20260809 22:51:21.480842 28804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:21.513304 28804 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:21.513823 28804 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:21.526844 28804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35a9983c8729422794a29f25edb413bf: Generated new TSK 0
I20260809 22:51:21.527963 28804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:21.543339 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28656
W20260809 22:51:21.544754 28764 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45065
W20260809 22:51:21.545562 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28656: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:21.545717 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 28656
2026-08-09T22:51:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2212 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:21 dist-test-slave-pww0 krb5kdc[28820](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:21 dist-test-slave-pww0 krb5kdc[28820](info): set up 2 sockets
Aug 09 22:51:21 dist-test-slave-pww0 krb5kdc[28820](info): commencing operation
krb5kdc: starting...
W20260809 22:51:22.832093 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:23 dist-test-slave-pww0 krb5kdc[28820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:23.162194 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:23.526209 28841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:23.526549 28841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:23.526675 28841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:23.534605 28841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:23.534759 28841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:23.534839 28841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:23.534934 28841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:23.534997 28841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:23.545492 28841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:23.546913 28841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:23.548349 28841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:23.560776 28851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:23.560776 28852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:23.560818 28858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:23.561380 28841 server_base.cc:1034] running on GCE node
I20260809 22:51:23.562399 28841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:23.564060 28841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:23.565281 28841 hybrid_clock.cc:648] HybridClock initialized: now 1786315883565226 us; error 54 us; skew 500 ppm
I20260809 22:51:23.565658 28841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:23 dist-test-slave-pww0 krb5kdc[28820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315883, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:23.573139 28841 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:23.574803 28841 webserver.cc:460] Webserver started at http://127.0.0.1:39527/ using document root <none> and password file <none>
I20260809 22:51:23.575589 28841 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:23.575727 28841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:23.576038 28841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:23.579154 28841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "5cdf5bdb75854ebea7d65144cfa0018e"
format_stamp: "Formatted at 2026-08-09 22:51:23 on dist-test-slave-pww0"
server_key: "9d7300518588938194e62200b9951108"
server_key_iv: "459d7a7a0d3378d841b52e4ada821d7c"
server_key_version: "encryptionkey@0"
I20260809 22:51:23.579980 28841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "5cdf5bdb75854ebea7d65144cfa0018e"
format_stamp: "Formatted at 2026-08-09 22:51:23 on dist-test-slave-pww0"
server_key: "9d7300518588938194e62200b9951108"
server_key_iv: "459d7a7a0d3378d841b52e4ada821d7c"
server_key_version: "encryptionkey@0"
I20260809 22:51:23.585418 28841 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:51:23.589211 28868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:23.590790 28841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 22:51:23.591032 28841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "5cdf5bdb75854ebea7d65144cfa0018e"
format_stamp: "Formatted at 2026-08-09 22:51:23 on dist-test-slave-pww0"
server_key: "9d7300518588938194e62200b9951108"
server_key_iv: "459d7a7a0d3378d841b52e4ada821d7c"
server_key_version: "encryptionkey@0"
I20260809 22:51:23.591264 28841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:23.611601 28841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:23.616165 28841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:23.616508 28841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:23.636276 28841 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42269
I20260809 22:51:23.636307 28960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42269 every 8 connection(s)
I20260809 22:51:23.637866 28841 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:23.641988 28962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:23.646517 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 28841
I20260809 22:51:23.646713 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:23.647294 25120 external_mini_cluster.cc:1428] Setting key b7592a7bafa2b9abbecc082a93bf3b22
I20260809 22:51:23.656046 28962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e: Bootstrap starting.
I20260809 22:51:23.660131 28962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:23.661530 28962 log.cc:824] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:51:23 dist-test-slave-pww0 krb5kdc[28820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:51:23.664758 28962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e: No bootstrap required, opened a new log
I20260809 22:51:23.671257 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:23.649253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51485 (local address 127.0.0.1:42269)
0809 22:51:23.650079 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:23.650092 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:23.650144 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:51:23.651821 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:23.651847 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:23.652297 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:23.652640 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:23.654532 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:23.655936 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:23.657758 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:23.658471 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:23.658602 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:23.663756 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:23.663764 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:23.663780 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:23.663824 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:23.666890 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:23.667661 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:23.667668 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:23.667675 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:23.667853 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:23.668247 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:23.668250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:23.668253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:23.668833 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:23.669160 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:23.669637 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 22:51:23.670113 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":374,"threads_started":1}
I20260809 22:51:23.672966 28962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5bdb75854ebea7d65144cfa0018e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } }
I20260809 22:51:23.673348 28962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:23.673451 28962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdf5bdb75854ebea7d65144cfa0018e, State: Initialized, Role: FOLLOWER
I20260809 22:51:23.674152 28962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5bdb75854ebea7d65144cfa0018e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } }
I20260809 22:51:23.674451 28962 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:23.674616 28962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:23.674820 28962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:23.675243 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:23.677237 28962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5bdb75854ebea7d65144cfa0018e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } }
I20260809 22:51:23.677774 28962 leader_election.cc:302] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cdf5bdb75854ebea7d65144cfa0018e; no voters:
I20260809 22:51:23.678455 28962 leader_election.cc:288] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:23.678552 28971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:23.680982 28962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:23.685199 28971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [term 1 LEADER]: Becoming Leader. State: Replica: 5cdf5bdb75854ebea7d65144cfa0018e, State: Running, Role: LEADER
I20260809 22:51:23.686049 28971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5bdb75854ebea7d65144cfa0018e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } }
I20260809 22:51:23.689446 28975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cdf5bdb75854ebea7d65144cfa0018e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5bdb75854ebea7d65144cfa0018e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } } }
I20260809 22:51:23.690423 28975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:23.690060 28976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cdf5bdb75854ebea7d65144cfa0018e. Latest consensus state: current_term: 1 leader_uuid: "5cdf5bdb75854ebea7d65144cfa0018e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdf5bdb75854ebea7d65144cfa0018e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42269 } } }
I20260809 22:51:23.691232 28976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:23.692596 28985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:23.699088 28985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:23.710078 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:23.679793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48983 (local address 127.0.0.1:42269)
0809 22:51:23.679983 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:23.679991 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:23.689147 (+ 9156us) server_negotiation.cc:407] Connection header received
0809 22:51:23.689495 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:23.689503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:23.689607 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:23.689700 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:23.692506 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:23.693685 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:23.695898 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:23.696731 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:23.696862 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:23.700793 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:23.700804 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:23.700809 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:23.700864 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:23.705123 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:23.706339 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:23.706345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:23.706350 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:23.706538 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:23.707055 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:23.707061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:23.707065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:23.707633 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:23.707950 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:23.709363 (+ 1413us) server_negotiation.cc:299] Negotiation successful
0809 22:51:23.709636 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:51:23.714253 28985 catalog_manager.cc:1347] Generated new cluster ID: 3ab0d7a9b16c4c058c10b21c2e2702e3
I20260809 22:51:23.714397 28985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:23.728031 28985 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:23.729403 28985 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:23.735647 28985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cdf5bdb75854ebea7d65144cfa0018e: Generated new TSK 0
I20260809 22:51:23.736518 28985 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:23.746222 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 28841
I20260809 22:51:23.747638 28954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:23.838865 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 28841
2026-08-09T22:51:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:23 dist-test-slave-pww0 krb5kdc[29010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:23 dist-test-slave-pww0 krb5kdc[29010](info): set up 2 sockets
Aug 09 22:51:23 dist-test-slave-pww0 krb5kdc[29010](info): commencing operation
krb5kdc: starting...
W20260809 22:51:25.132236 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.015s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:25 dist-test-slave-pww0 krb5kdc[29010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:25.468178 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:25.825368 29039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:25.825683 29039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:25.825829 29039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:25.833240 29039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:25.833395 29039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:25.833477 29039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:25.833563 29039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:25.833624 29039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:25.843894 29039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:25.845271 29039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:25.846647 29039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:25.859004 29045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:25.858950 29052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:25.859025 29050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:25.859918 29039 server_base.cc:1034] running on GCE node
I20260809 22:51:25.860801 29039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:25.862298 29039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:25.863545 29039 hybrid_clock.cc:648] HybridClock initialized: now 1786315885863499 us; error 35 us; skew 500 ppm
I20260809 22:51:25.863903 29039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:25 dist-test-slave-pww0 krb5kdc[29010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315885, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:25.867163 29039 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:25.870325 29039 webserver.cc:460] Webserver started at http://127.0.0.1:40651/ using document root <none> and password file <none>
I20260809 22:51:25.871014 29039 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:25.871117 29039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:25.871387 29039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:25.874537 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "fa7e39144af14a79b35566422928f03b"
format_stamp: "Formatted at 2026-08-09 22:51:25 on dist-test-slave-pww0"
server_key: "778dc726607ffbed45503c54482c9247"
server_key_iv: "2951efc40e20cd19b32c5332f14f8ca6"
server_key_version: "encryptionkey@0"
I20260809 22:51:25.875375 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "fa7e39144af14a79b35566422928f03b"
format_stamp: "Formatted at 2026-08-09 22:51:25 on dist-test-slave-pww0"
server_key: "778dc726607ffbed45503c54482c9247"
server_key_iv: "2951efc40e20cd19b32c5332f14f8ca6"
server_key_version: "encryptionkey@0"
I20260809 22:51:25.880887 29039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:51:25.884770 29061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:25.886271 29039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:25.886463 29039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "fa7e39144af14a79b35566422928f03b"
format_stamp: "Formatted at 2026-08-09 22:51:25 on dist-test-slave-pww0"
server_key: "778dc726607ffbed45503c54482c9247"
server_key_iv: "2951efc40e20cd19b32c5332f14f8ca6"
server_key_version: "encryptionkey@0"
I20260809 22:51:25.886665 29039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:25.912794 29039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:25.918129 29039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:25.918440 29039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:25.938067 29039 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35005
I20260809 22:51:25.938098 29159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35005 every 8 connection(s)
I20260809 22:51:25.939558 29039 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:25.941978 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 29039
I20260809 22:51:25.942188 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:25.942757 25120 external_mini_cluster.cc:1428] Setting key 5da7ed0c4a55d1c76f7a167e6206b86d
I20260809 22:51:25.944371 29160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:25.956907 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:25.944670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58697 (local address 127.0.0.1:35005)
0809 22:51:25.945508 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:25.945521 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:25.945563 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:51:25.947252 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:25.947276 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:25.947758 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:25.948116 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:25.948759 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:25.950138 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:25.952075 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:25.952824 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:25.952969 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:25.954309 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:25.954319 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:25.954336 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:25.954946 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:25.955031 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:25.955071 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:25.955179 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:25.955514 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 22:51:25.955860 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":351,"threads_started":1}
I20260809 22:51:25.958276 29160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b: Bootstrap starting.
I20260809 22:51:25.961130 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:25.961874 29160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:25.962924 29160 log.cc:824] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:25.966483 29160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b: No bootstrap required, opened a new log
I20260809 22:51:25.972533 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:25.965619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58691 (local address 127.0.0.1:35005)
0809 22:51:25.965825 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:25.965831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:25.966183 (+ 352us) server_negotiation.cc:407] Connection header received
0809 22:51:25.966395 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:25.966398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:25.966459 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:25.966544 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:25.967236 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:25.968107 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:25.969990 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:25.970478 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:25.970570 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:25.971806 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:25.971813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:25.971817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:25.971839 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:25.971879 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:25.971894 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:25.971967 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:25.972124 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 22:51:25.972208 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:51:25.976106 29160 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7e39144af14a79b35566422928f03b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35005 } }
I20260809 22:51:25.976526 29160 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:25.976639 29160 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa7e39144af14a79b35566422928f03b, State: Initialized, Role: FOLLOWER
I20260809 22:51:25.977300 29160 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7e39144af14a79b35566422928f03b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35005 } }
I20260809 22:51:25.977538 29160 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:25.977675 29160 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:25.977859 29160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:25.980562 29160 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7e39144af14a79b35566422928f03b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35005 } }
I20260809 22:51:25.981125 29160 leader_election.cc:302] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa7e39144af14a79b35566422928f03b; no voters:
I20260809 22:51:25.981751 29160 leader_election.cc:288] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:25.981889 29177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:25.982592 29177 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [term 1 LEADER]: Becoming Leader. State: Replica: fa7e39144af14a79b35566422928f03b, State: Running, Role: LEADER
I20260809 22:51:25.983140 29177 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7e39144af14a79b35566422928f03b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35005 } }
I20260809 22:51:25.983980 29160 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:25.985996 29179 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa7e39144af14a79b35566422928f03b. Latest consensus state: current_term: 1 leader_uuid: "fa7e39144af14a79b35566422928f03b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7e39144af14a79b35566422928f03b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35005 } } }
I20260809 22:51:25.986423 29179 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:25.986145 29178 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa7e39144af14a79b35566422928f03b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7e39144af14a79b35566422928f03b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35005 } } }
I20260809 22:51:25.986728 29178 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:25.990401 29185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:25.995330 29185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:26.005127 29185 catalog_manager.cc:1347] Generated new cluster ID: bcfed1819ad9474fb158bdb1a4f272a4
I20260809 22:51:26.005272 29185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:26.027108 29185 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:26.029150 29185 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:26.044525 29185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa7e39144af14a79b35566422928f03b: Generated new TSK 0
I20260809 22:51:26.045499 29185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:26.056591 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29039
W20260809 22:51:26.058159 29148 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58697
W20260809 22:51:26.058928 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29039: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:26.059067 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 29039
2026-08-09T22:51:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2222 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:26 dist-test-slave-pww0 krb5kdc[29212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:26 dist-test-slave-pww0 krb5kdc[29212](info): set up 2 sockets
Aug 09 22:51:26 dist-test-slave-pww0 krb5kdc[29212](info): commencing operation
krb5kdc: starting...
W20260809 22:51:27.343487 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:27 dist-test-slave-pww0 krb5kdc[29212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:27.683216 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40433
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:28.051750 29237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:28.052110 29237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:28.052268 29237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:28.059955 29237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:28.060109 29237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:28.060194 29237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:28.060288 29237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:28.060350 29237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:28.070854 29237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:28.072355 29237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:28.073726 29237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:28.086359 29249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:28.086427 29248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:28.086397 29251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:28.087208 29237 server_base.cc:1034] running on GCE node
I20260809 22:51:28.088109 29237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:28.089635 29237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:28.090859 29237 hybrid_clock.cc:648] HybridClock initialized: now 1786315888090806 us; error 44 us; skew 500 ppm
I20260809 22:51:28.091229 29237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:28 dist-test-slave-pww0 krb5kdc[29212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315888, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:28.094668 29237 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:28.096441 29237 webserver.cc:460] Webserver started at http://127.0.0.1:40705/ using document root <none> and password file <none>
I20260809 22:51:28.097205 29237 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:28.097332 29237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:28.097654 29237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:28.101003 29237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "f526fdff10ef443aa2efe950259a5045"
format_stamp: "Formatted at 2026-08-09 22:51:28 on dist-test-slave-pww0"
server_key: "544de158124a66102baa2b2d1e2cd9c7"
server_key_iv: "faa12193e7d87595382899848b5ed814"
server_key_version: "encryptionkey@0"
I20260809 22:51:28.101900 29237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "f526fdff10ef443aa2efe950259a5045"
format_stamp: "Formatted at 2026-08-09 22:51:28 on dist-test-slave-pww0"
server_key: "544de158124a66102baa2b2d1e2cd9c7"
server_key_iv: "faa12193e7d87595382899848b5ed814"
server_key_version: "encryptionkey@0"
I20260809 22:51:28.107785 29237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:51:28.111677 29263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:28.113171 29237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:28.113368 29237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "f526fdff10ef443aa2efe950259a5045"
format_stamp: "Formatted at 2026-08-09 22:51:28 on dist-test-slave-pww0"
server_key: "544de158124a66102baa2b2d1e2cd9c7"
server_key_iv: "faa12193e7d87595382899848b5ed814"
server_key_version: "encryptionkey@0"
I20260809 22:51:28.113577 29237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:28.119716 29237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:28.124185 29237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:28.124495 29237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:28.143191 29237 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40433
I20260809 22:51:28.143229 29364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40433 every 8 connection(s)
I20260809 22:51:28.144737 29237 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:28.148357 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 29237
I20260809 22:51:28.148515 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:28.148979 25120 external_mini_cluster.cc:1428] Setting key 7e67cb7238604c3a018001073406f3ed
I20260809 22:51:28.148993 29365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:28.158524 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:28.150899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44247 (local address 127.0.0.1:40433)
0809 22:51:28.151816 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:28.151837 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:28.151891 (+ 54us) server_negotiation.cc:407] Connection header received
0809 22:51:28.154158 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:28.154193 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:28.154726 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:28.155268 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:28.156398 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:28.156407 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:28.156429 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:28.156450 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:28.156527 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:28.156556 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:28.156805 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:28.156952 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 22:51:28.157188 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":338,"threads_started":1}
I20260809 22:51:28.162701 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:28.164863 29365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045: Bootstrap starting.
I20260809 22:51:28.169260 29365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:28.170626 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:28.167951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36903 (local address 127.0.0.1:40433)
0809 22:51:28.168164 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:28.168172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:28.168673 (+ 501us) server_negotiation.cc:407] Connection header received
0809 22:51:28.168861 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:28.168865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:28.168920 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:28.169003 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:28.169985 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:28.169989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:28.169992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:28.170007 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:28.170050 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:28.170066 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:28.170149 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:28.170289 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 22:51:28.170344 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:51:28.170703 29365 log.cc:824] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:28.173924 29365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045: No bootstrap required, opened a new log
I20260809 22:51:28.181946 29365 raft_consensus.cc:348] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f526fdff10ef443aa2efe950259a5045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40433 } }
I20260809 22:51:28.182372 29365 raft_consensus.cc:374] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:28.182479 29365 raft_consensus.cc:729] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f526fdff10ef443aa2efe950259a5045, State: Initialized, Role: FOLLOWER
I20260809 22:51:28.183106 29365 consensus_queue.cc:260] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f526fdff10ef443aa2efe950259a5045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40433 } }
I20260809 22:51:28.183378 29365 raft_consensus.cc:388] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:28.183552 29365 raft_consensus.cc:482] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:28.183718 29365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:28.186249 29365 raft_consensus.cc:504] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f526fdff10ef443aa2efe950259a5045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40433 } }
I20260809 22:51:28.186807 29365 leader_election.cc:302] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f526fdff10ef443aa2efe950259a5045; no voters:
I20260809 22:51:28.187381 29365 leader_election.cc:288] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:28.187557 29381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:28.188244 29381 raft_consensus.cc:686] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [term 1 LEADER]: Becoming Leader. State: Replica: f526fdff10ef443aa2efe950259a5045, State: Running, Role: LEADER
I20260809 22:51:28.188808 29381 consensus_queue.cc:237] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f526fdff10ef443aa2efe950259a5045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40433 } }
I20260809 22:51:28.189543 29365 sys_catalog.cc:564] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:28.191701 29383 sys_catalog.cc:455] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f526fdff10ef443aa2efe950259a5045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f526fdff10ef443aa2efe950259a5045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40433 } } }
I20260809 22:51:28.192108 29383 sys_catalog.cc:458] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:28.192191 29384 sys_catalog.cc:455] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f526fdff10ef443aa2efe950259a5045. Latest consensus state: current_term: 1 leader_uuid: "f526fdff10ef443aa2efe950259a5045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f526fdff10ef443aa2efe950259a5045" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40433 } } }
I20260809 22:51:28.192538 29384 sys_catalog.cc:458] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:28.197314 29399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:28.203226 29399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:28.213474 29399 catalog_manager.cc:1347] Generated new cluster ID: 61bfba7df10243b6bc73f568c53fe3aa
I20260809 22:51:28.213625 29399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:28.235778 29399 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:28.236258 29399 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:28.247934 29399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f526fdff10ef443aa2efe950259a5045: Generated new TSK 0
I20260809 22:51:28.248924 29399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:28.258747 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29237
W20260809 22:51:28.261343 29355 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44247
W20260809 22:51:28.262189 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29237: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:28.262382 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 29237
2026-08-09T22:51:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2209 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:28 dist-test-slave-pww0 krb5kdc[29424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:28 dist-test-slave-pww0 krb5kdc[29424](info): set up 2 sockets
Aug 09 22:51:28 dist-test-slave-pww0 krb5kdc[29424](info): commencing operation
krb5kdc: starting...
W20260809 22:51:29.571409 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:29 dist-test-slave-pww0 krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:29.911690 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:30.281394 29444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:30.281710 29444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:30.281807 29444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:30.289407 29444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:30.289543 29444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:30.289592 29444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:30.289654 29444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:30.289712 29444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:30.300105 29444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:30.301488 29444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:30.302870 29444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:30.315407 29457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:30.315493 29452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:30.315444 29453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:30.316237 29444 server_base.cc:1034] running on GCE node
I20260809 22:51:30.317332 29444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:30.318971 29444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:30.320194 29444 hybrid_clock.cc:648] HybridClock initialized: now 1786315890320133 us; error 61 us; skew 500 ppm
I20260809 22:51:30.320583 29444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:30 dist-test-slave-pww0 krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315890, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:30.324082 29444 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:30.325734 29444 webserver.cc:460] Webserver started at http://127.0.0.1:43361/ using document root <none> and password file <none>
I20260809 22:51:30.326432 29444 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:30.326548 29444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:30.326822 29444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:30.330039 29444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "7b37784b1f4e40f687124633dcdff558"
format_stamp: "Formatted at 2026-08-09 22:51:30 on dist-test-slave-pww0"
server_key: "fe5b3ee1bcce09be907b0f503abaa7af"
server_key_iv: "bf889825379590809b7c87d05689ab1d"
server_key_version: "encryptionkey@0"
I20260809 22:51:30.330845 29444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "7b37784b1f4e40f687124633dcdff558"
format_stamp: "Formatted at 2026-08-09 22:51:30 on dist-test-slave-pww0"
server_key: "fe5b3ee1bcce09be907b0f503abaa7af"
server_key_iv: "bf889825379590809b7c87d05689ab1d"
server_key_version: "encryptionkey@0"
I20260809 22:51:30.336426 29444 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 22:51:30.340251 29469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:30.341703 29444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:30.341885 29444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "7b37784b1f4e40f687124633dcdff558"
format_stamp: "Formatted at 2026-08-09 22:51:30 on dist-test-slave-pww0"
server_key: "fe5b3ee1bcce09be907b0f503abaa7af"
server_key_iv: "bf889825379590809b7c87d05689ab1d"
server_key_version: "encryptionkey@0"
I20260809 22:51:30.342099 29444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:30.368594 29444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:30.373404 29444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:30.373703 29444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:30.393421 29444 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43329
I20260809 22:51:30.393461 29555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43329 every 8 connection(s)
I20260809 22:51:30.394950 29444 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:30.396756 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 29444
I20260809 22:51:30.396943 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:30.397390 25120 external_mini_cluster.cc:1428] Setting key d47114cb96e42394ba51257a10908d85
I20260809 22:51:30.399778 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:51:30 dist-test-slave-pww0 krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:51:30.413817 29556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558: Bootstrap starting.
I20260809 22:51:30.417685 29556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:30.418804 29556 log.cc:824] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:30.421898 29556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558: No bootstrap required, opened a new log
I20260809 22:51:30.422382 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:30.399369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:48581 (local address 127.0.0.1:43329)
0809 22:51:30.400319 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:30.400333 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:30.400380 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:51:30.402195 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:30.402221 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:30.402693 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:30.403096 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:30.405054 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:30.406512 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:30.408299 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:30.409036 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:30.409197 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:30.414387 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:30.414402 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:30.414426 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:30.414490 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:30.417822 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:30.418776 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:30.418784 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:30.418795 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:30.418993 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:30.419541 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:30.419545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:30.419547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:30.420144 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:30.420478 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:30.421004 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 22:51:30.421327 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":425,"threads_started":1}
I20260809 22:51:30.426715 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:30.430637 29556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b37784b1f4e40f687124633dcdff558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260809 22:51:30.431108 29556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:30.431253 29556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b37784b1f4e40f687124633dcdff558, State: Initialized, Role: FOLLOWER
I20260809 22:51:30.432168 29556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b37784b1f4e40f687124633dcdff558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260809 22:51:30.432468 29556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:30.432638 29556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:30.432838 29556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:30.435827 29556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b37784b1f4e40f687124633dcdff558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260809 22:51:30.436532 29556 leader_election.cc:302] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b37784b1f4e40f687124633dcdff558; no voters:
I20260809 22:51:30.437337 29556 leader_election.cc:288] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:30.437510 29569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:30.439760 29556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:30.444011 29569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [term 1 LEADER]: Becoming Leader. State: Replica: 7b37784b1f4e40f687124633dcdff558, State: Running, Role: LEADER
I20260809 22:51:30.445329 29569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b37784b1f4e40f687124633dcdff558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260809 22:51:30.450142 29573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b37784b1f4e40f687124633dcdff558. Latest consensus state: current_term: 1 leader_uuid: "7b37784b1f4e40f687124633dcdff558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b37784b1f4e40f687124633dcdff558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } } }
I20260809 22:51:30.450717 29573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:30.451740 29595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:30.451592 29572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b37784b1f4e40f687124633dcdff558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b37784b1f4e40f687124633dcdff558" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } } }
I20260809 22:51:30.451941 29572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:30.452343 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:30.431373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:40585 (local address 127.0.0.1:43329)
0809 22:51:30.431608 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:30.431616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:30.432049 (+ 433us) server_negotiation.cc:407] Connection header received
0809 22:51:30.432240 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:30.432244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:30.432332 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:30.432430 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:30.434558 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:30.435349 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:30.437161 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:30.438049 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:30.439096 (+ 1047us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:30.442126 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:30.442135 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:30.442142 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:30.442194 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:30.446793 (+ 4599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:30.448173 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:30.448176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:30.448179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:30.448329 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:30.448936 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:30.448940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:30.448943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:30.449415 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:30.449781 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:30.451078 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0809 22:51:30.451310 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:51:30.459430 29595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:30.470309 29595 catalog_manager.cc:1347] Generated new cluster ID: f148179fdad34c14bd677f0ac063b3e4
I20260809 22:51:30.470443 29595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:30.484941 29595 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:30.486378 29595 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:30.496651 29595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b37784b1f4e40f687124633dcdff558: Generated new TSK 0
I20260809 22:51:30.497792 29595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:30.514668 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29444
I20260809 22:51:30.516638 29550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:30.611039 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 29444
W20260809 22:51:30.639350 29420 connection.cc:535] client connection to 127.0.0.1:43329 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43329): Connection reset by peer (error 104)
2026-08-09T22:51:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:30 dist-test-slave-pww0 krb5kdc[29612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:30 dist-test-slave-pww0 krb5kdc[29612](info): set up 2 sockets
Aug 09 22:51:30 dist-test-slave-pww0 krb5kdc[29612](info): commencing operation
krb5kdc: starting...
W20260809 22:51:31.897329 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:32 dist-test-slave-pww0 krb5kdc[29612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:32.238238 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38317
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:32.598145 29635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:32.598497 29635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:32.598670 29635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:32.606319 29635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:32.606487 29635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:32.606575 29635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:32.606669 29635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:32.606732 29635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:32.617439 29635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:32.618847 29635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:32.620318 29635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:32.632746 29646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:32.632791 29654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:32.632721 29648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:32.633533 29635 server_base.cc:1034] running on GCE node
I20260809 22:51:32.634428 29635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:32.636116 29635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:32.637359 29635 hybrid_clock.cc:648] HybridClock initialized: now 1786315892637350 us; error 84 us; skew 500 ppm
I20260809 22:51:32.637794 29635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:32 dist-test-slave-pww0 krb5kdc[29612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:32.641292 29635 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:32.642982 29635 webserver.cc:460] Webserver started at http://127.0.0.1:36111/ using document root <none> and password file <none>
I20260809 22:51:32.643774 29635 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:32.643903 29635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:32.644218 29635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:32.647421 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "ee37e6ecbe394b3292cbb6394bd8156e"
format_stamp: "Formatted at 2026-08-09 22:51:32 on dist-test-slave-pww0"
server_key: "aa3d6784560b4e72a06738ae75807b03"
server_key_iv: "c10427a59b4dea8c33d32b3b502ae7f9"
server_key_version: "encryptionkey@0"
I20260809 22:51:32.648334 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "ee37e6ecbe394b3292cbb6394bd8156e"
format_stamp: "Formatted at 2026-08-09 22:51:32 on dist-test-slave-pww0"
server_key: "aa3d6784560b4e72a06738ae75807b03"
server_key_iv: "c10427a59b4dea8c33d32b3b502ae7f9"
server_key_version: "encryptionkey@0"
I20260809 22:51:32.654456 29635 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 22:51:32.658521 29665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:32.660175 29635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:51:32.660377 29635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "ee37e6ecbe394b3292cbb6394bd8156e"
format_stamp: "Formatted at 2026-08-09 22:51:32 on dist-test-slave-pww0"
server_key: "aa3d6784560b4e72a06738ae75807b03"
server_key_iv: "c10427a59b4dea8c33d32b3b502ae7f9"
server_key_version: "encryptionkey@0"
I20260809 22:51:32.660581 29635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:32.674527 29635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:32.679383 29635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:32.679749 29635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:32.698664 29635 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38317
I20260809 22:51:32.698698 29757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38317 every 8 connection(s)
I20260809 22:51:32.700186 29635 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:32.702560 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 29635
I20260809 22:51:32.702746 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:32.703202 25120 external_mini_cluster.cc:1428] Setting key 80174dae7c2164588a4d12845faa5129
I20260809 22:51:32.704511 29758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:32.719272 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:32.705070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:41039 (local address 127.0.0.1:38317)
0809 22:51:32.705872 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:32.705887 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:32.705942 (+ 55us) server_negotiation.cc:407] Connection header received
0809 22:51:32.708104 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:32.708146 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:32.708734 (+ 588us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:32.709206 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:32.709794 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:32.712015 (+ 2221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:32.713831 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:32.715020 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:32.716015 (+ 995us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:32.716805 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:32.716814 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:32.716831 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:32.717446 (+ 615us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:32.717531 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:32.717561 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:32.717738 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:32.718038 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 22:51:32.718231 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":342,"threads_started":1}
I20260809 22:51:32.719969 29758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e: Bootstrap starting.
I20260809 22:51:32.723785 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:32.723904 29758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:32.725033 29758 log.cc:824] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:32.728435 29758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e: No bootstrap required, opened a new log
I20260809 22:51:32.735705 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:32.728590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:54039 (local address 127.0.0.1:38317)
0809 22:51:32.728773 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:32.728780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:32.729165 (+ 385us) server_negotiation.cc:407] Connection header received
0809 22:51:32.729355 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:32.729360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:32.729441 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:32.729557 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:32.730085 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:32.731315 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:32.732732 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:32.733446 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:32.733593 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:32.734744 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:32.734751 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:32.734756 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:32.734781 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:32.734836 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:32.734857 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:32.734977 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:32.735210 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 22:51:32.735283 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:51:32.738405 29758 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee37e6ecbe394b3292cbb6394bd8156e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38317 } }
I20260809 22:51:32.738828 29758 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:32.738933 29758 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee37e6ecbe394b3292cbb6394bd8156e, State: Initialized, Role: FOLLOWER
I20260809 22:51:32.739626 29758 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee37e6ecbe394b3292cbb6394bd8156e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38317 } }
I20260809 22:51:32.739882 29758 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:32.740025 29758 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:32.740195 29758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:32.742527 29758 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee37e6ecbe394b3292cbb6394bd8156e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38317 } }
I20260809 22:51:32.743175 29758 leader_election.cc:302] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee37e6ecbe394b3292cbb6394bd8156e; no voters:
I20260809 22:51:32.743934 29758 leader_election.cc:288] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:32.744046 29772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:32.744840 29772 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [term 1 LEADER]: Becoming Leader. State: Replica: ee37e6ecbe394b3292cbb6394bd8156e, State: Running, Role: LEADER
I20260809 22:51:32.745429 29772 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee37e6ecbe394b3292cbb6394bd8156e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38317 } }
I20260809 22:51:32.746174 29758 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:32.748373 29774 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee37e6ecbe394b3292cbb6394bd8156e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee37e6ecbe394b3292cbb6394bd8156e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38317 } } }
I20260809 22:51:32.749305 29774 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:32.748975 29776 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee37e6ecbe394b3292cbb6394bd8156e. Latest consensus state: current_term: 1 leader_uuid: "ee37e6ecbe394b3292cbb6394bd8156e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee37e6ecbe394b3292cbb6394bd8156e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38317 } } }
I20260809 22:51:32.749837 29776 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:32.751019 29786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:32.759339 29786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:32.769522 29786 catalog_manager.cc:1347] Generated new cluster ID: 657ce6e3455f4fe79ac3c6c63319c899
I20260809 22:51:32.769661 29786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:32.801661 29786 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:32.803768 29786 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:32.813416 29786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee37e6ecbe394b3292cbb6394bd8156e: Generated new TSK 0
I20260809 22:51:32.814643 29786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:32.822721 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29635
W20260809 22:51:32.824110 29750 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.195:41039
W20260809 22:51:32.825100 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29635: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:32.825249 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 29635
W20260809 22:51:32.849530 29609 connection.cc:535] client connection to 127.0.0.1:38317 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38317): Connection reset by peer (error 104)
2026-08-09T22:51:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2209 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:32 dist-test-slave-pww0 krb5kdc[29816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:32 dist-test-slave-pww0 krb5kdc[29816](info): set up 2 sockets
Aug 09 22:51:32 dist-test-slave-pww0 krb5kdc[29816](info): commencing operation
krb5kdc: starting...
W20260809 22:51:34.110321 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:34 dist-test-slave-pww0 krb5kdc[29816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:34.454489 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:34.821704 29837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:34.822036 29837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:34.822178 29837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:34.829622 29837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:34.829785 29837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:34.829864 29837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:34.829946 29837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:34.830006 29837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:34.840374 29837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:34.841795 29837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:34.843158 29837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:34.855381 29850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:34.855356 29859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:34.855305 29851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:34.856758 29837 server_base.cc:1034] running on GCE node
I20260809 22:51:34.857684 29837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:34.859272 29837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:34.860505 29837 hybrid_clock.cc:648] HybridClock initialized: now 1786315894860450 us; error 53 us; skew 500 ppm
I20260809 22:51:34.860898 29837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:34 dist-test-slave-pww0 krb5kdc[29816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315894, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:34.868929 29837 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:34.870714 29837 webserver.cc:460] Webserver started at http://127.0.0.1:45749/ using document root <none> and password file <none>
I20260809 22:51:34.871515 29837 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:34.871660 29837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:34.871995 29837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:34.875205 29837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "5dc888cdabcc46da8dd5f3110f9936e1"
format_stamp: "Formatted at 2026-08-09 22:51:34 on dist-test-slave-pww0"
server_key: "1eb6291b6a851b2d222d5f4b074619ea"
server_key_iv: "458abc60328259b3c59ff7c895318ba7"
server_key_version: "encryptionkey@0"
I20260809 22:51:34.876075 29837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "5dc888cdabcc46da8dd5f3110f9936e1"
format_stamp: "Formatted at 2026-08-09 22:51:34 on dist-test-slave-pww0"
server_key: "1eb6291b6a851b2d222d5f4b074619ea"
server_key_iv: "458abc60328259b3c59ff7c895318ba7"
server_key_version: "encryptionkey@0"
I20260809 22:51:34.881587 29837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:51:34.885426 29875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:34.886881 29837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:51:34.887082 29837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "5dc888cdabcc46da8dd5f3110f9936e1"
format_stamp: "Formatted at 2026-08-09 22:51:34 on dist-test-slave-pww0"
server_key: "1eb6291b6a851b2d222d5f4b074619ea"
server_key_iv: "458abc60328259b3c59ff7c895318ba7"
server_key_version: "encryptionkey@0"
I20260809 22:51:34.887316 29837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:34.893791 29837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:34.898216 29837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:34.898519 29837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:34.917407 29837 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37215
I20260809 22:51:34.917436 29964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37215 every 8 connection(s)
I20260809 22:51:34.918943 29837 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:34.919489 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 29837
I20260809 22:51:34.919673 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:34.920282 25120 external_mini_cluster.cc:1428] Setting key 349c033140af3107080775612d6c33c0
I20260809 22:51:34.924196 29965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:34.929936 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:34.922311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:41015 (local address 127.0.0.1:37215)
0809 22:51:34.923216 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:34.923231 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:34.923286 (+ 55us) server_negotiation.cc:407] Connection header received
0809 22:51:34.925728 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:34.925761 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:34.926232 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:34.926728 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:34.927810 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:34.927822 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:34.927840 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:34.927858 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:34.927963 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:34.927991 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:34.928235 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:34.928380 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 22:51:34.928618 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":356,"threads_started":1}
I20260809 22:51:34.934245 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:34.937903 29965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1: Bootstrap starting.
I20260809 22:51:34.941479 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:34.938698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:60383 (local address 127.0.0.1:37215)
0809 22:51:34.938930 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:34.938937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:34.939360 (+ 423us) server_negotiation.cc:407] Connection header received
0809 22:51:34.939592 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:34.939597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:34.939659 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:34.939773 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:34.940818 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:34.940822 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:34.940826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:34.940841 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:34.940884 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:34.940900 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:34.940990 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:34.941132 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 22:51:34.941185 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:51:34.942122 29965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:34.943357 29965 log.cc:824] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:34.946347 29965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1: No bootstrap required, opened a new log
I20260809 22:51:34.954175 29965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc888cdabcc46da8dd5f3110f9936e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37215 } }
I20260809 22:51:34.954610 29965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:34.954712 29965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc888cdabcc46da8dd5f3110f9936e1, State: Initialized, Role: FOLLOWER
I20260809 22:51:34.955655 29965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc888cdabcc46da8dd5f3110f9936e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37215 } }
I20260809 22:51:34.955946 29965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:34.956115 29965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:34.956313 29965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:34.958734 29965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc888cdabcc46da8dd5f3110f9936e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37215 } }
I20260809 22:51:34.959311 29965 leader_election.cc:302] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dc888cdabcc46da8dd5f3110f9936e1; no voters:
I20260809 22:51:34.959959 29965 leader_election.cc:288] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:34.960121 29985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:34.960762 29985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [term 1 LEADER]: Becoming Leader. State: Replica: 5dc888cdabcc46da8dd5f3110f9936e1, State: Running, Role: LEADER
I20260809 22:51:34.961328 29985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc888cdabcc46da8dd5f3110f9936e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37215 } }
I20260809 22:51:34.962131 29965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:34.964394 29987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dc888cdabcc46da8dd5f3110f9936e1. Latest consensus state: current_term: 1 leader_uuid: "5dc888cdabcc46da8dd5f3110f9936e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc888cdabcc46da8dd5f3110f9936e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37215 } } }
I20260809 22:51:34.964859 29987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:34.965392 29986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dc888cdabcc46da8dd5f3110f9936e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc888cdabcc46da8dd5f3110f9936e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37215 } } }
I20260809 22:51:34.965665 29986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:34.967103 29997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:34.972415 29997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:34.981293 29997 catalog_manager.cc:1347] Generated new cluster ID: f9e10b181cc3472b89bd16c7fe273559
I20260809 22:51:34.981433 29997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:34.999233 29997 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:34.999764 29997 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:35.007937 29997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dc888cdabcc46da8dd5f3110f9936e1: Generated new TSK 0
I20260809 22:51:35.008909 29997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:35.016119 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29837
W20260809 22:51:35.017966 29955 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.195:41015
W20260809 22:51:35.018808 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 29837: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:35.018954 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 29837
2026-08-09T22:51:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2199 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:35 dist-test-slave-pww0 krb5kdc[30022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:35 dist-test-slave-pww0 krb5kdc[30022](info): set up 2 sockets
Aug 09 22:51:35 dist-test-slave-pww0 krb5kdc[30022](info): commencing operation
krb5kdc: starting...
W20260809 22:51:36.319175 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:36 dist-test-slave-pww0 krb5kdc[30022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:36.658949 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:37.023010 30042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:37.023360 30042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:37.023488 30042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:37.031069 30042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:37.031198 30042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:37.031243 30042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:37.031322 30042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:37.031365 30042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:37.041922 30042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43809
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:37.043291 30042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:37.044675 30042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:37.057087 30051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:37.057130 30052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:37.057309 30058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:37.058178 30042 server_base.cc:1034] running on GCE node
I20260809 22:51:37.059027 30042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:37.060698 30042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:37.061918 30042 hybrid_clock.cc:648] HybridClock initialized: now 1786315897061864 us; error 54 us; skew 500 ppm
I20260809 22:51:37.062305 30042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:37 dist-test-slave-pww0 krb5kdc[30022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315897, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:37.065687 30042 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:37.067363 30042 webserver.cc:460] Webserver started at http://127.0.0.1:38369/ using document root <none> and password file <none>
I20260809 22:51:37.068131 30042 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:37.068289 30042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:37.068609 30042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:37.071877 30042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b"
format_stamp: "Formatted at 2026-08-09 22:51:37 on dist-test-slave-pww0"
server_key: "61e6057aa57eefd2566b85d8de817496"
server_key_iv: "54c5c859741fa9653d6af5d15d9fa2c1"
server_key_version: "encryptionkey@0"
I20260809 22:51:37.072710 30042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b"
format_stamp: "Formatted at 2026-08-09 22:51:37 on dist-test-slave-pww0"
server_key: "61e6057aa57eefd2566b85d8de817496"
server_key_iv: "54c5c859741fa9653d6af5d15d9fa2c1"
server_key_version: "encryptionkey@0"
I20260809 22:51:37.078430 30042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:51:37.082322 30069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:37.083910 30042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:51:37.084107 30042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b"
format_stamp: "Formatted at 2026-08-09 22:51:37 on dist-test-slave-pww0"
server_key: "61e6057aa57eefd2566b85d8de817496"
server_key_iv: "54c5c859741fa9653d6af5d15d9fa2c1"
server_key_version: "encryptionkey@0"
I20260809 22:51:37.084327 30042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:37.116962 30042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:37.121624 30042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:37.121924 30042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:37.141795 30042 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43809
I20260809 22:51:37.141836 30162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43809 every 8 connection(s)
I20260809 22:51:37.143304 30042 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:37.147939 30163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:37.153548 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 30042
I20260809 22:51:37.153721 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:37.154176 25120 external_mini_cluster.cc:1428] Setting key 4bcc2f508f54c5f87c41aff2f4ab5ebc
I20260809 22:51:37.161844 30163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b: Bootstrap starting.
I20260809 22:51:37.165876 30163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:37.167083 30163 log.cc:824] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:37.170205 30163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b: No bootstrap required, opened a new log
Aug 09 22:51:37 dist-test-slave-pww0 krb5kdc[30022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:51:37.178304 30163 raft_consensus.cc:348] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } }
I20260809 22:51:37.178797 30163 raft_consensus.cc:374] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:37.179190 30163 raft_consensus.cc:729] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e80cc1aa6cbe43ccaf3de6fb457e743b, State: Initialized, Role: FOLLOWER
I20260809 22:51:37.179872 30163 consensus_queue.cc:260] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } }
I20260809 22:51:37.180030 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:37.156309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:49429 (local address 127.0.0.1:43809)
0809 22:51:37.157224 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:37.157241 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:37.157302 (+ 61us) server_negotiation.cc:407] Connection header received
0809 22:51:37.159893 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:37.159931 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:37.160550 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:37.161036 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:37.162500 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:37.164562 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:37.166084 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:37.166875 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:37.167085 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:37.171712 (+ 4627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:37.171720 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:37.171737 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:37.171784 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:37.175213 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:37.176175 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:37.176183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:37.176191 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:37.176370 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:37.176869 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:37.176873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:37.176875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:37.177521 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:37.177840 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:37.178399 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 22:51:37.178767 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":337,"threads_started":1}
I20260809 22:51:37.180124 30163 raft_consensus.cc:388] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:37.180248 30163 raft_consensus.cc:482] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:37.180445 30163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:37.183425 30163 raft_consensus.cc:504] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } }
I20260809 22:51:37.184109 30163 leader_election.cc:302] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e80cc1aa6cbe43ccaf3de6fb457e743b; no voters:
I20260809 22:51:37.184602 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:37.184814 30163 leader_election.cc:288] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:37.184933 30171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:37.187294 30163 sys_catalog.cc:564] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:37.192148 30171 raft_consensus.cc:686] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [term 1 LEADER]: Becoming Leader. State: Replica: e80cc1aa6cbe43ccaf3de6fb457e743b, State: Running, Role: LEADER
I20260809 22:51:37.193006 30171 consensus_queue.cc:237] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } }
I20260809 22:51:37.197793 30178 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } } }
I20260809 22:51:37.198231 30178 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:37.197764 30180 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e80cc1aa6cbe43ccaf3de6fb457e743b. Latest consensus state: current_term: 1 leader_uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e80cc1aa6cbe43ccaf3de6fb457e743b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } } }
I20260809 22:51:37.198704 30180 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:37.200289 30204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:37.207072 30204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:37.216161 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:37.194499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:43139 (local address 127.0.0.1:43809)
0809 22:51:37.195165 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:37.195182 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:37.195852 (+ 670us) server_negotiation.cc:407] Connection header received
0809 22:51:37.196105 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:37.196115 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:37.196205 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:37.196314 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:37.200878 (+ 4564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:37.201697 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:37.203289 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:37.204013 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:37.204139 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:37.207872 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:37.207881 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:37.207886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:37.207938 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:37.211977 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:37.213229 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:37.213235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:37.213238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:37.213407 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:37.214109 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:37.214115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:37.214119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:37.214556 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:37.214828 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:37.215570 (+ 742us) server_negotiation.cc:299] Negotiation successful
0809 22:51:37.215794 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517}
I20260809 22:51:37.221304 30204 catalog_manager.cc:1347] Generated new cluster ID: dff9cc3de5a944d4824e8c7f5a4dfa16
I20260809 22:51:37.221437 30204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:37.232169 30204 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:37.234166 30204 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:37.248366 30204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e80cc1aa6cbe43ccaf3de6fb457e743b: Generated new TSK 0
I20260809 22:51:37.249541 30204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:37.266418 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30042
I20260809 22:51:37.267832 30156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:37.361645 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 30042
W20260809 22:51:37.391083 30020 connection.cc:535] client connection to 127.0.0.1:43809 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43809): Connection reset by peer (error 104)
2026-08-09T22:51:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:37 dist-test-slave-pww0 krb5kdc[30219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:37 dist-test-slave-pww0 krb5kdc[30219](info): set up 2 sockets
Aug 09 22:51:37 dist-test-slave-pww0 krb5kdc[30219](info): commencing operation
krb5kdc: starting...
W20260809 22:51:38.647800 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:38 dist-test-slave-pww0 krb5kdc[30219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:38.982970 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:39.364987 30242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:39.365304 30242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:39.365439 30242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:39.373014 30242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:39.373157 30242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:39.373238 30242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:39.373325 30242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:39.373380 30242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:39.384191 30242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36815
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:39.385547 30242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:39.387375 30242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:39.399672 30251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:39.399699 30248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:39.399876 30249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:39.400462 30242 server_base.cc:1034] running on GCE node
I20260809 22:51:39.401252 30242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:39.402737 30242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:39.404017 30242 hybrid_clock.cc:648] HybridClock initialized: now 1786315899403953 us; error 48 us; skew 500 ppm
I20260809 22:51:39.404383 30242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:39 dist-test-slave-pww0 krb5kdc[30219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315899, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:39.407564 30242 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:39.409245 30242 webserver.cc:460] Webserver started at http://127.0.0.1:32795/ using document root <none> and password file <none>
I20260809 22:51:39.409983 30242 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:39.410108 30242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:39.410430 30242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:39.413654 30242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "8462586e033d41029cd21a475ed8e523"
format_stamp: "Formatted at 2026-08-09 22:51:39 on dist-test-slave-pww0"
server_key: "1e08bcfd3150be3009414539e6287700"
server_key_iv: "ed108afff9943e2a73bd0d7fedd13842"
server_key_version: "encryptionkey@0"
I20260809 22:51:39.414462 30242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "8462586e033d41029cd21a475ed8e523"
format_stamp: "Formatted at 2026-08-09 22:51:39 on dist-test-slave-pww0"
server_key: "1e08bcfd3150be3009414539e6287700"
server_key_iv: "ed108afff9943e2a73bd0d7fedd13842"
server_key_version: "encryptionkey@0"
I20260809 22:51:39.420066 30242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:51:39.423828 30266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:39.425283 30242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:39.425464 30242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "8462586e033d41029cd21a475ed8e523"
format_stamp: "Formatted at 2026-08-09 22:51:39 on dist-test-slave-pww0"
server_key: "1e08bcfd3150be3009414539e6287700"
server_key_iv: "ed108afff9943e2a73bd0d7fedd13842"
server_key_version: "encryptionkey@0"
I20260809 22:51:39.425678 30242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:39.464813 30242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:39.469406 30242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:39.469703 30242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:39.488348 30242 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36815
I20260809 22:51:39.488380 30357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36815 every 8 connection(s)
I20260809 22:51:39.489854 30242 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:39.490541 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 30242
I20260809 22:51:39.490753 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:39.491319 25120 external_mini_cluster.cc:1428] Setting key 342296d71b7a941a236b6f13cc025d2a
I20260809 22:51:39.494530 30358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:39.505472 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:39.493162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:54403 (local address 127.0.0.1:36815)
0809 22:51:39.493964 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:39.493976 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:39.494017 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:51:39.495773 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:39.495799 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:39.496215 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:39.496556 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:39.497225 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:39.498608 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:39.500555 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:39.501298 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:39.501453 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:39.502834 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:39.502842 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:39.502859 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:39.503522 (+ 663us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:39.503605 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:39.503632 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:39.503785 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:39.504069 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 22:51:39.504285 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":348,"threads_started":1}
I20260809 22:51:39.507853 30358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523: Bootstrap starting.
I20260809 22:51:39.509829 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:39.511535 30358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:39.513023 30358 log.cc:824] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:39.516592 30358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523: No bootstrap required, opened a new log
I20260809 22:51:39.522383 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:39.515418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:49915 (local address 127.0.0.1:36815)
0809 22:51:39.515649 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:39.515657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:39.516222 (+ 565us) server_negotiation.cc:407] Connection header received
0809 22:51:39.516395 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:39.516400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:39.516483 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:39.516619 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:39.517203 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:39.518321 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:39.519668 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:39.520374 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:39.520506 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:39.521594 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:39.521602 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:39.521606 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:39.521630 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:39.521686 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:39.521709 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:39.521829 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:39.521981 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 22:51:39.522039 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 22:51:39.525739 30358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8462586e033d41029cd21a475ed8e523" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36815 } }
I20260809 22:51:39.526153 30358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:39.526269 30358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8462586e033d41029cd21a475ed8e523, State: Initialized, Role: FOLLOWER
I20260809 22:51:39.526870 30358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8462586e033d41029cd21a475ed8e523" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36815 } }
I20260809 22:51:39.527096 30358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:39.527200 30358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:39.527330 30358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:39.529947 30358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8462586e033d41029cd21a475ed8e523" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36815 } }
I20260809 22:51:39.530520 30358 leader_election.cc:302] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8462586e033d41029cd21a475ed8e523; no voters:
I20260809 22:51:39.531091 30358 leader_election.cc:288] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:39.531219 30374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:39.531935 30374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [term 1 LEADER]: Becoming Leader. State: Replica: 8462586e033d41029cd21a475ed8e523, State: Running, Role: LEADER
I20260809 22:51:39.532544 30374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8462586e033d41029cd21a475ed8e523" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36815 } }
I20260809 22:51:39.533140 30358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:39.536093 30378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8462586e033d41029cd21a475ed8e523. Latest consensus state: current_term: 1 leader_uuid: "8462586e033d41029cd21a475ed8e523" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8462586e033d41029cd21a475ed8e523" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36815 } } }
I20260809 22:51:39.536119 30377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8462586e033d41029cd21a475ed8e523" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8462586e033d41029cd21a475ed8e523" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36815 } } }
I20260809 22:51:39.536573 30378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:39.536612 30377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:39.537669 30384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:39.543130 30384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:39.552790 30384 catalog_manager.cc:1347] Generated new cluster ID: f4c03dce50a54a3fb38eadf043fb8d3a
I20260809 22:51:39.552927 30384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:39.573037 30384 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:39.574406 30384 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:39.582922 30384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8462586e033d41029cd21a475ed8e523: Generated new TSK 0
I20260809 22:51:39.583847 30384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:39.594703 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30242
W20260809 22:51:39.596589 30348 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.195:54403
W20260809 22:51:39.597536 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30242: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:39.597736 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 30242
W20260809 22:51:39.624747 30216 connection.cc:535] client connection to 127.0.0.1:36815 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36815): Connection reset by peer (error 104)
2026-08-09T22:51:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2234 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:39 dist-test-slave-pww0 krb5kdc[30402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:39 dist-test-slave-pww0 krb5kdc[30402](info): set up 2 sockets
Aug 09 22:51:39 dist-test-slave-pww0 krb5kdc[30402](info): commencing operation
krb5kdc: starting...
W20260809 22:51:40.902362 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:41 dist-test-slave-pww0 krb5kdc[30402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:51:41.246888 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38081
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:41.615056 30429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:41.615384 30429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:41.615536 30429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:41.623040 30429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:41.623175 30429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:41.623225 30429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:41.623294 30429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:41.623339 30429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:41.633711 30429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38081
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:41.635506 30429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:41.636983 30429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:41.649638 30438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:41.649796 30439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:41.649685 30441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:41.650664 30429 server_base.cc:1034] running on GCE node
I20260809 22:51:41.651517 30429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:41.653126 30429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:41.654371 30429 hybrid_clock.cc:648] HybridClock initialized: now 1786315901654313 us; error 52 us; skew 500 ppm
I20260809 22:51:41.654753 30429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:41 dist-test-slave-pww0 krb5kdc[30402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315901, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:41.661736 30429 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:51:41.663431 30429 webserver.cc:460] Webserver started at http://127.0.0.1:42671/ using document root <none> and password file <none>
I20260809 22:51:41.664219 30429 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:41.664353 30429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:41.664677 30429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:41.667829 30429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "9b9e01eca5a5438bbf35be2377f738d5"
format_stamp: "Formatted at 2026-08-09 22:51:41 on dist-test-slave-pww0"
server_key: "3a7d3faf9a252304fcb93b0963212b04"
server_key_iv: "2d6fe30f86a327e418f7f53d12d32a5a"
server_key_version: "encryptionkey@0"
I20260809 22:51:41.668623 30429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "9b9e01eca5a5438bbf35be2377f738d5"
format_stamp: "Formatted at 2026-08-09 22:51:41 on dist-test-slave-pww0"
server_key: "3a7d3faf9a252304fcb93b0963212b04"
server_key_iv: "2d6fe30f86a327e418f7f53d12d32a5a"
server_key_version: "encryptionkey@0"
I20260809 22:51:41.674203 30429 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 22:51:41.677947 30454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:41.679325 30429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:51:41.679549 30429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "9b9e01eca5a5438bbf35be2377f738d5"
format_stamp: "Formatted at 2026-08-09 22:51:41 on dist-test-slave-pww0"
server_key: "3a7d3faf9a252304fcb93b0963212b04"
server_key_iv: "2d6fe30f86a327e418f7f53d12d32a5a"
server_key_version: "encryptionkey@0"
I20260809 22:51:41.679751 30429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:41.685649 30429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:41.689961 30429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:41.690263 30429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:41.708534 30429 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38081
I20260809 22:51:41.708572 30546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38081 every 8 connection(s)
I20260809 22:51:41.710039 30429 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:41.711090 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 30429
I20260809 22:51:41.711262 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:41.711753 25120 external_mini_cluster.cc:1428] Setting key 10571585b00f092ed6931123490b012e
I20260809 22:51:41.714641 30549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:41.721019 30554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:41.713707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:42153 (local address 127.0.0.1:38081)
0809 22:51:41.714518 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:41.714535 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:41.714589 (+ 54us) server_negotiation.cc:407] Connection header received
0809 22:51:41.716934 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:41.716966 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:41.717405 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:41.717889 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:41.718905 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:41.718913 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:41.718930 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:41.718949 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:41.719041 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:41.719075 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:41.719328 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:41.719496 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 22:51:41.719714 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":302,"threads_started":1}
I20260809 22:51:41.724972 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:41.729250 30549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5: Bootstrap starting.
I20260809 22:51:41.732969 30554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:41.730574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:57309 (local address 127.0.0.1:38081)
0809 22:51:41.730773 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:41.730780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:41.731091 (+ 311us) server_negotiation.cc:407] Connection header received
0809 22:51:41.731241 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:41.731247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:41.731316 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:41.731450 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:41.732347 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:41.732352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:41.732358 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:41.732377 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:41.732428 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:41.732449 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:41.732568 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:41.732614 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 22:51:41.732672 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:51:41.733333 30549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:41.734745 30549 log.cc:824] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:41.737807 30549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5: No bootstrap required, opened a new log
I20260809 22:51:41.746001 30549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9e01eca5a5438bbf35be2377f738d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } }
I20260809 22:51:41.746527 30549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:41.746634 30549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b9e01eca5a5438bbf35be2377f738d5, State: Initialized, Role: FOLLOWER
I20260809 22:51:41.747372 30549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9e01eca5a5438bbf35be2377f738d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } }
I20260809 22:51:41.747704 30549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:41.747880 30549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:41.748083 30549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:41.750403 30549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9e01eca5a5438bbf35be2377f738d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } }
I20260809 22:51:41.750979 30549 leader_election.cc:302] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b9e01eca5a5438bbf35be2377f738d5; no voters:
I20260809 22:51:41.751673 30549 leader_election.cc:288] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:41.751796 30563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:41.752471 30563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [term 1 LEADER]: Becoming Leader. State: Replica: 9b9e01eca5a5438bbf35be2377f738d5, State: Running, Role: LEADER
I20260809 22:51:41.753079 30563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9e01eca5a5438bbf35be2377f738d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } }
I20260809 22:51:41.753854 30549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:41.756145 30568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b9e01eca5a5438bbf35be2377f738d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9e01eca5a5438bbf35be2377f738d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } } }
I20260809 22:51:41.757107 30568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:41.756773 30571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b9e01eca5a5438bbf35be2377f738d5. Latest consensus state: current_term: 1 leader_uuid: "9b9e01eca5a5438bbf35be2377f738d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b9e01eca5a5438bbf35be2377f738d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38081 } } }
I20260809 22:51:41.757627 30571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:41.760535 30582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:41.766430 30582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:41.776569 30582 catalog_manager.cc:1347] Generated new cluster ID: bd5f465fb0134b3b8ea7cbbf2e6d587a
I20260809 22:51:41.776762 30582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:41.787761 30582 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:41.788224 30582 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:41.797696 30582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b9e01eca5a5438bbf35be2377f738d5: Generated new TSK 0
I20260809 22:51:41.798619 30582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:41.810847 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30429
W20260809 22:51:41.812144 30540 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.195:42153
W20260809 22:51:41.812989 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30429: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:41.813141 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 30429
2026-08-09T22:51:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2213 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:41 dist-test-slave-pww0 krb5kdc[30612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:41 dist-test-slave-pww0 krb5kdc[30612](info): set up 2 sockets
Aug 09 22:51:41 dist-test-slave-pww0 krb5kdc[30612](info): commencing operation
krb5kdc: starting...
W20260809 22:51:43.115546 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:43 dist-test-slave-pww0 krb5kdc[30612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:51:43.462944 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38649
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:43.825143 30630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:43.825474 30630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:43.825578 30630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:43.833045 30630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:43.833191 30630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:43.833248 30630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:43.833313 30630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:43.833357 30630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:43.843701 30630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38649
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:43.845084 30630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:43.846421 30630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:43.858554 30642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:43.858706 30640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:43.858575 30639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:43.859709 30630 server_base.cc:1034] running on GCE node
I20260809 22:51:43.860551 30630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:43.862112 30630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:43.863355 30630 hybrid_clock.cc:648] HybridClock initialized: now 1786315903863299 us; error 53 us; skew 500 ppm
I20260809 22:51:43.863788 30630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:43 dist-test-slave-pww0 krb5kdc[30612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315903, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:43.867100 30630 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:51:43.868793 30630 webserver.cc:460] Webserver started at http://127.24.136.62:41045/ using document root <none> and password file <none>
I20260809 22:51:43.869534 30630 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:43.869647 30630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:43.869915 30630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:43.873155 30630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "631d7e8821ed447681127eb3604ca4cd"
format_stamp: "Formatted at 2026-08-09 22:51:43 on dist-test-slave-pww0"
server_key: "db915f63c007f2d7d5ea8a2b60397897"
server_key_iv: "226ef1fd5d45d8813e0a182fc8c7054c"
server_key_version: "encryptionkey@0"
I20260809 22:51:43.873922 30630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "631d7e8821ed447681127eb3604ca4cd"
format_stamp: "Formatted at 2026-08-09 22:51:43 on dist-test-slave-pww0"
server_key: "db915f63c007f2d7d5ea8a2b60397897"
server_key_iv: "226ef1fd5d45d8813e0a182fc8c7054c"
server_key_version: "encryptionkey@0"
I20260809 22:51:43.879619 30630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260809 22:51:43.883389 30654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:43.884924 30630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:51:43.885113 30630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "631d7e8821ed447681127eb3604ca4cd"
format_stamp: "Formatted at 2026-08-09 22:51:43 on dist-test-slave-pww0"
server_key: "db915f63c007f2d7d5ea8a2b60397897"
server_key_iv: "226ef1fd5d45d8813e0a182fc8c7054c"
server_key_version: "encryptionkey@0"
I20260809 22:51:43.885322 30630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:43.898547 30630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:43.903072 30630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:43.903370 30630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:43.922888 30630 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38649
I20260809 22:51:43.922919 30739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38649 every 8 connection(s)
I20260809 22:51:43.924461 30630 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:43.928629 30741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:43.929134 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 30630
I20260809 22:51:43.929303 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:43.929741 25120 external_mini_cluster.cc:1428] Setting key f1bb7549ea2dd8fdffc0a0014a1352bd
I20260809 22:51:43.943688 30741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd: Bootstrap starting.
Aug 09 22:51:43 dist-test-slave-pww0 krb5kdc[30612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:43.947105 30741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:43.948678 30741 log.cc:824] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:43.952443 30741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd: No bootstrap required, opened a new log
I20260809 22:51:43.956137 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:43.931770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36097 (local address 127.24.136.62:38649)
0809 22:51:43.932626 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:43.932643 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:43.932696 (+ 53us) server_negotiation.cc:407] Connection header received
0809 22:51:43.935421 (+ 2725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:43.935473 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:43.936057 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:43.936544 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:43.937953 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:43.940041 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:43.941446 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:43.942576 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:43.942775 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:43.947070 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:43.947087 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:43.947109 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:43.947163 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:43.950949 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:43.951887 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:43.951897 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:43.951906 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:43.952130 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:43.952587 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:43.952592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:43.952595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:43.953385 (+ 790us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:43.953722 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:43.954196 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 22:51:43.954610 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":342,"threads_started":1}
I20260809 22:51:43.960410 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:43.961831 30741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631d7e8821ed447681127eb3604ca4cd" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38649 } }
I20260809 22:51:43.962227 30741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:43.962348 30741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 631d7e8821ed447681127eb3604ca4cd, State: Initialized, Role: FOLLOWER
I20260809 22:51:43.962934 30741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631d7e8821ed447681127eb3604ca4cd" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38649 } }
I20260809 22:51:43.963189 30741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:43.963349 30741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:43.963608 30741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:43.966543 30741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631d7e8821ed447681127eb3604ca4cd" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38649 } }
I20260809 22:51:43.967278 30741 leader_election.cc:302] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 631d7e8821ed447681127eb3604ca4cd; no voters:
I20260809 22:51:43.968019 30741 leader_election.cc:288] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:43.968171 30765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:43.968850 30765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [term 1 LEADER]: Becoming Leader. State: Replica: 631d7e8821ed447681127eb3604ca4cd, State: Running, Role: LEADER
I20260809 22:51:43.969564 30765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631d7e8821ed447681127eb3604ca4cd" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38649 } }
I20260809 22:51:43.971741 30765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 631d7e8821ed447681127eb3604ca4cd. Latest consensus state: current_term: 1 leader_uuid: "631d7e8821ed447681127eb3604ca4cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631d7e8821ed447681127eb3604ca4cd" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38649 } } }
I20260809 22:51:43.973161 30765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:43.973704 30770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "631d7e8821ed447681127eb3604ca4cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631d7e8821ed447681127eb3604ca4cd" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38649 } } }
I20260809 22:51:43.973958 30770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:43.976408 30741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:43.978576 30773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:43.984629 30773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:43.984747 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:43.964923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41265 (local address 127.24.136.62:38649)
0809 22:51:43.965147 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:43.965155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:43.965645 (+ 490us) server_negotiation.cc:407] Connection header received
0809 22:51:43.965844 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:43.965847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:43.965915 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:43.965999 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:43.968010 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:43.969242 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:43.971059 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:43.971947 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:43.972844 (+ 897us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:43.975584 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:43.975593 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:43.975599 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:43.975647 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:43.979745 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:43.980947 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:43.980953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:43.980956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:43.981283 (+ 327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:43.981964 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:43.981970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:43.981974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:43.982440 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:43.982787 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:43.984069 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0809 22:51:43.984303 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:51:43.997166 30773 catalog_manager.cc:1347] Generated new cluster ID: d31aa0f447f64fc79a2e234fd945c752
I20260809 22:51:43.997314 30773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:44.009847 30773 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:44.011438 30773 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:44.019264 30773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 631d7e8821ed447681127eb3604ca4cd: Generated new TSK 0
I20260809 22:51:44.020538 30773 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:44.026190 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30630
I20260809 22:51:44.027922 30733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:44.120395 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 30630
W20260809 22:51:44.143043 30609 connection.cc:535] client connection to 127.24.136.62:38649 recv error: Network error: failed to read from TLS socket (remote: 127.24.136.62:38649): Connection reset by peer (error 104)
2026-08-09T22:51:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:44 dist-test-slave-pww0 krb5kdc[30811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:44 dist-test-slave-pww0 krb5kdc[30811](info): set up 2 sockets
Aug 09 22:51:44 dist-test-slave-pww0 krb5kdc[30811](info): commencing operation
krb5kdc: starting...
W20260809 22:51:45.408319 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:45 dist-test-slave-pww0 krb5kdc[30811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:51:45.764848 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38433
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:38731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:46.124428 30835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:46.124825 30835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:46.124954 30835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:46.132495 30835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:46.132634 30835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:46.132699 30835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:46.132800 30835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:46.132858 30835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:46.143337 30835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:38731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38433
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:46.144790 30835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:46.146250 30835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:46.159035 30844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:46.159108 30842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:46.159103 30841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:46.160450 30835 server_base.cc:1034] running on GCE node
I20260809 22:51:46.161355 30835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:46.162941 30835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:46.164223 30835 hybrid_clock.cc:648] HybridClock initialized: now 1786315906164167 us; error 46 us; skew 500 ppm
I20260809 22:51:46.164657 30835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:46 dist-test-slave-pww0 krb5kdc[30811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315906, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:46.168133 30835 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:51:46.170110 30835 webserver.cc:460] Webserver started at http://127.24.136.62:36567/ using document root <none> and password file <none>
I20260809 22:51:46.170912 30835 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:46.171054 30835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:46.171401 30835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:46.174830 30835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "732c3c7b8ae44df09c27d528a4212484"
format_stamp: "Formatted at 2026-08-09 22:51:46 on dist-test-slave-pww0"
server_key: "56d1073fb35df7b01963dd6b4dff1d63"
server_key_iv: "82d908a61f2c7cec728acf095f6129fc"
server_key_version: "encryptionkey@0"
I20260809 22:51:46.175777 30835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "732c3c7b8ae44df09c27d528a4212484"
format_stamp: "Formatted at 2026-08-09 22:51:46 on dist-test-slave-pww0"
server_key: "56d1073fb35df7b01963dd6b4dff1d63"
server_key_iv: "82d908a61f2c7cec728acf095f6129fc"
server_key_version: "encryptionkey@0"
I20260809 22:51:46.181926 30835 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 22:51:46.186050 30853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:46.187654 30835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:51:46.187865 30835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "732c3c7b8ae44df09c27d528a4212484"
format_stamp: "Formatted at 2026-08-09 22:51:46 on dist-test-slave-pww0"
server_key: "56d1073fb35df7b01963dd6b4dff1d63"
server_key_iv: "82d908a61f2c7cec728acf095f6129fc"
server_key_version: "encryptionkey@0"
I20260809 22:51:46.188082 30835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:46.202272 30835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:46.206741 30835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:46.207046 30835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:46.225906 30835 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38433
I20260809 22:51:46.225939 30938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38433 every 8 connection(s)
I20260809 22:51:46.227509 30835 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:46.229110 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 30835
I20260809 22:51:46.229315 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:46.229781 25120 external_mini_cluster.cc:1428] Setting key 7cfb2d159977dd9a3349f74167d53749
I20260809 22:51:46.232228 30940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:46.244505 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:46.231708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59533 (local address 127.24.136.62:38433)
0809 22:51:46.232516 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:46.232532 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:46.232577 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:51:46.234277 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:46.234302 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:46.234723 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:46.235117 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:46.235827 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:46.237204 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:46.239144 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:46.239998 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:46.240141 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:46.241633 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:46.241643 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:46.241660 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:46.242327 (+ 667us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:46.242429 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:46.242468 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:46.242700 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:46.243026 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 22:51:46.243273 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":346,"threads_started":1}
I20260809 22:51:46.245875 30940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484: Bootstrap starting.
I20260809 22:51:46.249238 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:46.249614 30940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:46.250747 30940 log.cc:824] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:46.254411 30940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484: No bootstrap required, opened a new log
I20260809 22:51:46.260423 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:46.253997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55287 (local address 127.24.136.62:38433)
0809 22:51:46.254193 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:46.254199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:46.254590 (+ 391us) server_negotiation.cc:407] Connection header received
0809 22:51:46.254801 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:46.254805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:46.254877 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:46.254966 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:46.255604 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:46.256410 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:46.257749 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:46.258209 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:46.258348 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:46.259627 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:46.259631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:46.259635 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:46.259658 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:46.259703 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:46.259719 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:46.259829 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:46.260066 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 22:51:46.260130 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:51:46.262921 30940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c3c7b8ae44df09c27d528a4212484" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38433 } }
I20260809 22:51:46.263342 30940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:46.263445 30940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732c3c7b8ae44df09c27d528a4212484, State: Initialized, Role: FOLLOWER
I20260809 22:51:46.264166 30940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c3c7b8ae44df09c27d528a4212484" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38433 } }
I20260809 22:51:46.264489 30940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:46.264650 30940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:46.264807 30940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:46.267163 30940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c3c7b8ae44df09c27d528a4212484" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38433 } }
I20260809 22:51:46.267874 30940 leader_election.cc:302] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 732c3c7b8ae44df09c27d528a4212484; no voters:
I20260809 22:51:46.268492 30940 leader_election.cc:288] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:46.268628 30963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:46.269302 30963 raft_consensus.cc:686] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [term 1 LEADER]: Becoming Leader. State: Replica: 732c3c7b8ae44df09c27d528a4212484, State: Running, Role: LEADER
I20260809 22:51:46.269912 30963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c3c7b8ae44df09c27d528a4212484" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38433 } }
I20260809 22:51:46.270696 30940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:46.272923 30969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 732c3c7b8ae44df09c27d528a4212484. Latest consensus state: current_term: 1 leader_uuid: "732c3c7b8ae44df09c27d528a4212484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c3c7b8ae44df09c27d528a4212484" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38433 } } }
I20260809 22:51:46.273373 30969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:46.275120 30968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "732c3c7b8ae44df09c27d528a4212484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "732c3c7b8ae44df09c27d528a4212484" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38433 } } }
I20260809 22:51:46.275496 30968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:46.276690 30983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:46.282476 30983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:46.293437 30983 catalog_manager.cc:1347] Generated new cluster ID: 2f85b12e407c4b9b99ae134af08365ca
I20260809 22:51:46.293574 30983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:46.303090 30983 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:46.304515 30983 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:46.319239 30983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 732c3c7b8ae44df09c27d528a4212484: Generated new TSK 0
I20260809 22:51:46.320227 30983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:46.325934 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30835
W20260809 22:51:46.327977 30931 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59533
W20260809 22:51:46.328801 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 30835: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:46.328991 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 30835
W20260809 22:51:46.350839 30805 connection.cc:535] client connection to 127.24.136.62:38433 recv error: Network error: failed to read from TLS socket (remote: 127.24.136.62:38433): Connection reset by peer (error 104)
2026-08-09T22:51:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2207 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:46 dist-test-slave-pww0 krb5kdc[31002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:46 dist-test-slave-pww0 krb5kdc[31002](info): set up 2 sockets
Aug 09 22:51:46 dist-test-slave-pww0 krb5kdc[31002](info): commencing operation
krb5kdc: starting...
W20260809 22:51:47.611135 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:47 dist-test-slave-pww0 krb5kdc[31002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:51:47.955628 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:43509
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:38373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:43509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:48.324364 31025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:48.324681 31025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:48.324819 31025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:48.332222 31025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:48.332348 31025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:48.332424 31025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:48.332496 31025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:48.332557 31025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:48.342792 31025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:38373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:43509
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:43509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:48.344198 31025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:48.345541 31025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:48.357573 31036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:48.357616 31040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:48.357645 31038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:48.358486 31025 server_base.cc:1034] running on GCE node
I20260809 22:51:48.359222 31025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:48.360735 31025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:48.361938 31025 hybrid_clock.cc:648] HybridClock initialized: now 1786315908361877 us; error 62 us; skew 500 ppm
I20260809 22:51:48.362268 31025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:48 dist-test-slave-pww0 krb5kdc[31002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315908, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:48.369414 31025 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:51:48.371047 31025 webserver.cc:460] Webserver started at http://127.24.136.62:39313/ using document root <none> and password file <none>
I20260809 22:51:48.371819 31025 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:48.371948 31025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:48.372259 31025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:48.375349 31025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "7ca60014b18647ee86cfe0852b61bae8"
format_stamp: "Formatted at 2026-08-09 22:51:48 on dist-test-slave-pww0"
server_key: "6160da196a66f7cf3c83ceaf9647f326"
server_key_iv: "6c7ce7e9cfcd572526c1cb9305e20e37"
server_key_version: "encryptionkey@0"
I20260809 22:51:48.376152 31025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "7ca60014b18647ee86cfe0852b61bae8"
format_stamp: "Formatted at 2026-08-09 22:51:48 on dist-test-slave-pww0"
server_key: "6160da196a66f7cf3c83ceaf9647f326"
server_key_iv: "6c7ce7e9cfcd572526c1cb9305e20e37"
server_key_version: "encryptionkey@0"
I20260809 22:51:48.381439 31025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:51:48.385051 31053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:48.386442 31025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:51:48.386620 31025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "7ca60014b18647ee86cfe0852b61bae8"
format_stamp: "Formatted at 2026-08-09 22:51:48 on dist-test-slave-pww0"
server_key: "6160da196a66f7cf3c83ceaf9647f326"
server_key_iv: "6c7ce7e9cfcd572526c1cb9305e20e37"
server_key_version: "encryptionkey@0"
I20260809 22:51:48.386821 31025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:48.392789 31025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:48.397127 31025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:48.397428 31025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:48.415908 31025 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:43509
I20260809 22:51:48.415961 31137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:43509 every 8 connection(s)
I20260809 22:51:48.417493 31025 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:48.420325 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31025
I20260809 22:51:48.420476 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:48.420938 25120 external_mini_cluster.cc:1428] Setting key 4b4af033404cdde516a9e485bc6dd90c
I20260809 22:51:48.421561 31139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:48.428882 31146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:48.422785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39357 (local address 127.24.136.62:43509)
0809 22:51:48.423658 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:48.423671 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:48.423716 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:51:48.425253 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:48.425280 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:48.425642 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:48.426033 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:48.427117 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:48.427124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:48.427138 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:48.427158 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:48.427242 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:48.427269 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:48.427530 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:48.427686 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 22:51:48.427901 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":347,"threads_started":1}
I20260809 22:51:48.432819 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:48.435109 31139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8: Bootstrap starting.
I20260809 22:51:48.438789 31139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:48.439792 31146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:48.437096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58523 (local address 127.24.136.62:43509)
0809 22:51:48.437312 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:48.437319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:48.437780 (+ 461us) server_negotiation.cc:407] Connection header received
0809 22:51:48.437977 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:48.437981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:48.438031 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:48.438116 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:48.439106 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:48.439110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:48.439113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:48.439129 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:48.439175 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:48.439191 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:48.439292 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:48.439414 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 22:51:48.439488 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:51:48.440249 31139 log.cc:824] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:48.443110 31139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8: No bootstrap required, opened a new log
I20260809 22:51:48.450546 31139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca60014b18647ee86cfe0852b61bae8" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43509 } }
I20260809 22:51:48.450987 31139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:48.451086 31139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca60014b18647ee86cfe0852b61bae8, State: Initialized, Role: FOLLOWER
I20260809 22:51:48.451756 31139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca60014b18647ee86cfe0852b61bae8" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43509 } }
I20260809 22:51:48.452005 31139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:48.452167 31139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:48.452384 31139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:48.454707 31139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca60014b18647ee86cfe0852b61bae8" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43509 } }
I20260809 22:51:48.455247 31139 leader_election.cc:302] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ca60014b18647ee86cfe0852b61bae8; no voters:
I20260809 22:51:48.455930 31139 leader_election.cc:288] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:48.456073 31157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:48.456729 31157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [term 1 LEADER]: Becoming Leader. State: Replica: 7ca60014b18647ee86cfe0852b61bae8, State: Running, Role: LEADER
I20260809 22:51:48.457268 31157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca60014b18647ee86cfe0852b61bae8" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43509 } }
I20260809 22:51:48.458082 31139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:48.460134 31168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ca60014b18647ee86cfe0852b61bae8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca60014b18647ee86cfe0852b61bae8" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43509 } } }
I20260809 22:51:48.460583 31168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:48.461444 31169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ca60014b18647ee86cfe0852b61bae8. Latest consensus state: current_term: 1 leader_uuid: "7ca60014b18647ee86cfe0852b61bae8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ca60014b18647ee86cfe0852b61bae8" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 43509 } } }
I20260809 22:51:48.461860 31169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:48.463357 31178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:48.468150 31178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:48.477495 31178 catalog_manager.cc:1347] Generated new cluster ID: 886fd4582718405a81e3e7b1e926aef6
I20260809 22:51:48.477620 31178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:48.492328 31178 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:48.492846 31178 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:48.504681 31178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ca60014b18647ee86cfe0852b61bae8: Generated new TSK 0
I20260809 22:51:48.505609 31178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:48.510792 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31025
W20260809 22:51:48.512575 31130 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39357
W20260809 22:51:48.513337 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31025: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:48.513518 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31025
2026-08-09T22:51:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2192 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:48 dist-test-slave-pww0 krb5kdc[31201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:48 dist-test-slave-pww0 krb5kdc[31201](info): set up 2 sockets
Aug 09 22:51:48 dist-test-slave-pww0 krb5kdc[31201](info): commencing operation
krb5kdc: starting...
W20260809 22:51:49.814085 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:50 dist-test-slave-pww0 krb5kdc[31201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:51:50.159191 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38771
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:39313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:38771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:50.520671 31225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:50.521026 31225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:50.521164 31225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:50.528708 31225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:50.528846 31225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:50.528923 31225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:50.529002 31225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:50.529062 31225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:50.539554 31225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:39313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:38771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:38771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:50.541021 31225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:50.542408 31225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:50.555018 31233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:50.555187 31235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:50.554953 31237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:50.556293 31225 server_base.cc:1034] running on GCE node
I20260809 22:51:50.557155 31225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:50.558741 31225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:50.559955 31225 hybrid_clock.cc:648] HybridClock initialized: now 1786315910559901 us; error 51 us; skew 500 ppm
I20260809 22:51:50.560312 31225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:50 dist-test-slave-pww0 krb5kdc[31201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315910, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:50.563650 31225 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:51:50.565388 31225 webserver.cc:460] Webserver started at http://127.24.136.62:34953/ using document root <none> and password file <none>
I20260809 22:51:50.566150 31225 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:50.566282 31225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:50.566593 31225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:50.569916 31225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "88ae9119c53d4f4cb22f3d41caaf5351"
format_stamp: "Formatted at 2026-08-09 22:51:50 on dist-test-slave-pww0"
server_key: "76bde2f01ca17aaf7b974c29a9086796"
server_key_iv: "b19ab8c25dfc79d4d95e2e96301399d2"
server_key_version: "encryptionkey@0"
I20260809 22:51:50.570789 31225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "88ae9119c53d4f4cb22f3d41caaf5351"
format_stamp: "Formatted at 2026-08-09 22:51:50 on dist-test-slave-pww0"
server_key: "76bde2f01ca17aaf7b974c29a9086796"
server_key_iv: "b19ab8c25dfc79d4d95e2e96301399d2"
server_key_version: "encryptionkey@0"
I20260809 22:51:50.576862 31225 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 22:51:50.580875 31246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:50.582554 31225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:50.582791 31225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "88ae9119c53d4f4cb22f3d41caaf5351"
format_stamp: "Formatted at 2026-08-09 22:51:50 on dist-test-slave-pww0"
server_key: "76bde2f01ca17aaf7b974c29a9086796"
server_key_iv: "b19ab8c25dfc79d4d95e2e96301399d2"
server_key_version: "encryptionkey@0"
I20260809 22:51:50.582999 31225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:50.612787 31225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:50.617341 31225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:50.617650 31225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:50.636356 31225 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:38771
I20260809 22:51:50.636382 31345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:38771 every 8 connection(s)
I20260809 22:51:50.637848 31225 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:50.641902 31346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:50.644375 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31225
I20260809 22:51:50.644567 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:50.645141 25120 external_mini_cluster.cc:1428] Setting key 5c97c8da368b508551bd660383224dbc
I20260809 22:51:50.655158 31346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351: Bootstrap starting.
I20260809 22:51:50.659324 31346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351: Neither blocks nor log segments found. Creating new log.
Aug 09 22:51:50 dist-test-slave-pww0 krb5kdc[31201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:50.660606 31346 log.cc:824] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:50.664028 31346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351: No bootstrap required, opened a new log
I20260809 22:51:50.669405 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:50.647107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38025 (local address 127.24.136.62:38771)
0809 22:51:50.648016 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:50.648029 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:50.648077 (+ 48us) server_negotiation.cc:407] Connection header received
0809 22:51:50.649833 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:50.649858 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:50.650314 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:50.650665 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:50.652567 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:50.653998 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:50.655857 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:50.656560 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:50.656702 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:50.661725 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:50.661736 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:50.661762 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:50.661839 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:50.665021 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:50.665972 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:50.665979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:50.665989 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:50.666170 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:50.666568 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:50.666571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:50.666574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:50.667156 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:50.667412 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:50.667869 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 22:51:50.668282 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":396,"threads_started":1}
I20260809 22:51:50.672827 31346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ae9119c53d4f4cb22f3d41caaf5351" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38771 } }
I20260809 22:51:50.673211 31346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:50.673292 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:50.673321 31346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ae9119c53d4f4cb22f3d41caaf5351, State: Initialized, Role: FOLLOWER
I20260809 22:51:50.673949 31346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ae9119c53d4f4cb22f3d41caaf5351" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38771 } }
I20260809 22:51:50.674250 31346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:50.674386 31346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:50.674548 31346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:50.677814 31346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ae9119c53d4f4cb22f3d41caaf5351" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38771 } }
I20260809 22:51:50.678478 31346 leader_election.cc:302] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88ae9119c53d4f4cb22f3d41caaf5351; no voters:
I20260809 22:51:50.679136 31346 leader_election.cc:288] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:50.679251 31365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:50.681906 31346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:50.686151 31365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [term 1 LEADER]: Becoming Leader. State: Replica: 88ae9119c53d4f4cb22f3d41caaf5351, State: Running, Role: LEADER
I20260809 22:51:50.686883 31365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ae9119c53d4f4cb22f3d41caaf5351" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38771 } }
I20260809 22:51:50.690845 31366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88ae9119c53d4f4cb22f3d41caaf5351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ae9119c53d4f4cb22f3d41caaf5351" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38771 } } }
I20260809 22:51:50.690999 31367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88ae9119c53d4f4cb22f3d41caaf5351. Latest consensus state: current_term: 1 leader_uuid: "88ae9119c53d4f4cb22f3d41caaf5351" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ae9119c53d4f4cb22f3d41caaf5351" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 38771 } } }
I20260809 22:51:50.691437 31367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:50.691439 31366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:50.693476 31372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:50.694752 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:50.678452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33283 (local address 127.24.136.62:38771)
0809 22:51:50.678664 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:50.678670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:50.679209 (+ 539us) server_negotiation.cc:407] Connection header received
0809 22:51:50.679531 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:50.679536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:50.679600 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:50.679687 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:50.681418 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:50.682227 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:50.683628 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:50.684132 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:50.684250 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:50.686693 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:50.686700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:50.686704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:50.686743 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:50.690311 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:50.691294 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:50.691299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:50.691303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:50.691500 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:50.692030 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:50.692035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:50.692037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:50.692612 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:50.692953 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:50.694103 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0809 22:51:50.694361 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:51:50.704459 31372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:50.714288 31372 catalog_manager.cc:1347] Generated new cluster ID: 14923217cb4049c7a459250cccbe43c8
I20260809 22:51:50.714419 31372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:50.740995 31372 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:50.742551 31372 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:50.749135 31372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88ae9119c53d4f4cb22f3d41caaf5351: Generated new TSK 0
I20260809 22:51:50.750053 31372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:50.765908 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31225
I20260809 22:51:50.767099 31336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:51:50.862267 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31225
2026-08-09T22:51:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:50 dist-test-slave-pww0 krb5kdc[31398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:50 dist-test-slave-pww0 krb5kdc[31398](info): set up 2 sockets
Aug 09 22:51:50 dist-test-slave-pww0 krb5kdc[31398](info): commencing operation
krb5kdc: starting...
W20260809 22:51:52.158336 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:52 dist-test-slave-pww0 krb5kdc[31398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:51:52.503180 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45885
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:52.869660 31419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:52.869990 31419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:52.870119 31419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:52.877767 31419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:52.877904 31419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:52.877961 31419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:52.878052 31419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:52.878104 31419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:52.888667 31419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:52.890097 31419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:52.891583 31419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:52.904160 31426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:52.904139 31431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:52.904088 31429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:52.910671 31419 server_base.cc:1034] running on GCE node
I20260809 22:51:52.911742 31419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:52.913429 31419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:52.914673 31419 hybrid_clock.cc:648] HybridClock initialized: now 1786315912914621 us; error 49 us; skew 500 ppm
I20260809 22:51:52.915055 31419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:52 dist-test-slave-pww0 krb5kdc[31398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315912, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:52.918448 31419 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:51:52.920367 31419 webserver.cc:460] Webserver started at http://127.24.136.62:33123/ using document root <none> and password file <none>
I20260809 22:51:52.921218 31419 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:52.921362 31419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:52.921710 31419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:52.925052 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "db37d6c998854137bb1432546b737cfa"
format_stamp: "Formatted at 2026-08-09 22:51:52 on dist-test-slave-pww0"
server_key: "7334fa9eeec2a7217f84a11ae8dc3d24"
server_key_iv: "7cf546808fd96cf233df93d543c1a0c6"
server_key_version: "encryptionkey@0"
I20260809 22:51:52.925983 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "db37d6c998854137bb1432546b737cfa"
format_stamp: "Formatted at 2026-08-09 22:51:52 on dist-test-slave-pww0"
server_key: "7334fa9eeec2a7217f84a11ae8dc3d24"
server_key_iv: "7cf546808fd96cf233df93d543c1a0c6"
server_key_version: "encryptionkey@0"
I20260809 22:51:52.931864 31419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:51:52.935968 31444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:52.937631 31419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:52.937855 31419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "db37d6c998854137bb1432546b737cfa"
format_stamp: "Formatted at 2026-08-09 22:51:52 on dist-test-slave-pww0"
server_key: "7334fa9eeec2a7217f84a11ae8dc3d24"
server_key_iv: "7cf546808fd96cf233df93d543c1a0c6"
server_key_version: "encryptionkey@0"
I20260809 22:51:52.938079 31419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:52.956588 31419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:52.961203 31419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:52.961515 31419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:52.981426 31419 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:45885
I20260809 22:51:52.981432 31540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:45885 every 8 connection(s)
I20260809 22:51:52.982950 31419 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:52.987200 31541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:52.988628 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31419
I20260809 22:51:52.988834 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:52.989435 25120 external_mini_cluster.cc:1428] Setting key 591ed0b4c4e88d0b55ae8b30c2f6170e
I20260809 22:51:53.003098 31541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa: Bootstrap starting.
I20260809 22:51:53.004895 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:52.991532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60827 (local address 127.24.136.62:45885)
0809 22:51:52.992467 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:52.992481 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:52.992530 (+ 49us) server_negotiation.cc:407] Connection header received
0809 22:51:52.994307 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:52.994341 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:52.994854 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:52.995256 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:52.996020 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:52.997389 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:52.999601 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:53.000461 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:53.000632 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:53.002111 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:53.002120 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:53.002138 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:53.002796 (+ 658us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:53.002897 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:53.002927 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:53.003027 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:53.003389 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 22:51:53.003741 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":410,"threads_started":1}
I20260809 22:51:53.007227 31541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:53.008657 31541 log.cc:824] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:53.009262 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:53.011937 31541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa: No bootstrap required, opened a new log
I20260809 22:51:53.020790 31541 raft_consensus.cc:348] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37d6c998854137bb1432546b737cfa" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45885 } }
I20260809 22:51:53.021248 31541 raft_consensus.cc:374] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:53.021406 31541 raft_consensus.cc:729] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db37d6c998854137bb1432546b737cfa, State: Initialized, Role: FOLLOWER
I20260809 22:51:53.021533 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:53.013962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56499 (local address 127.24.136.62:45885)
0809 22:51:53.014154 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:53.014159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:53.015406 (+ 1247us) server_negotiation.cc:407] Connection header received
0809 22:51:53.015624 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:53.015628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:53.015705 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:53.015811 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:53.016403 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:53.017213 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:53.018660 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:53.019301 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:53.019508 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:53.020702 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:53.020708 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:53.020711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:53.020733 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:53.020784 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:53.020800 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:53.020884 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:53.021038 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 22:51:53.021149 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:51:53.022096 31541 consensus_queue.cc:260] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37d6c998854137bb1432546b737cfa" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45885 } }
I20260809 22:51:53.022418 31541 raft_consensus.cc:388] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:53.022575 31541 raft_consensus.cc:482] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:53.022789 31541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:53.025364 31541 raft_consensus.cc:504] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37d6c998854137bb1432546b737cfa" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45885 } }
I20260809 22:51:53.026063 31541 leader_election.cc:302] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db37d6c998854137bb1432546b737cfa; no voters:
I20260809 22:51:53.026806 31541 leader_election.cc:288] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:53.026990 31556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:53.027755 31556 raft_consensus.cc:686] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [term 1 LEADER]: Becoming Leader. State: Replica: db37d6c998854137bb1432546b737cfa, State: Running, Role: LEADER
I20260809 22:51:53.028393 31556 consensus_queue.cc:237] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37d6c998854137bb1432546b737cfa" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45885 } }
I20260809 22:51:53.029155 31541 sys_catalog.cc:564] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:53.031337 31557 sys_catalog.cc:455] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db37d6c998854137bb1432546b737cfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37d6c998854137bb1432546b737cfa" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45885 } } }
I20260809 22:51:53.031721 31557 sys_catalog.cc:458] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:53.032637 31562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:53.032558 31559 sys_catalog.cc:455] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader db37d6c998854137bb1432546b737cfa. Latest consensus state: current_term: 1 leader_uuid: "db37d6c998854137bb1432546b737cfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37d6c998854137bb1432546b737cfa" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45885 } } }
I20260809 22:51:53.033087 31559 sys_catalog.cc:458] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:53.038633 31562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:53.049304 31562 catalog_manager.cc:1347] Generated new cluster ID: 03e1a1d77cdb4c5a9aa8d04c50e7c84d
I20260809 22:51:53.049448 31562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:53.081844 31562 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:53.083238 31562 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:53.091749 31562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db37d6c998854137bb1432546b737cfa: Generated new TSK 0
I20260809 22:51:53.092746 31562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:53.112946 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31419
W20260809 22:51:53.114246 31530 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60827
W20260809 22:51:53.115063 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31419: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:53.115245 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31419
2026-08-09T22:51:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:53 dist-test-slave-pww0 krb5kdc[31585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:53 dist-test-slave-pww0 krb5kdc[31585](info): set up 2 sockets
Aug 09 22:51:53 dist-test-slave-pww0 krb5kdc[31585](info): commencing operation
krb5kdc: starting...
W20260809 22:51:54.415309 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:54 dist-test-slave-pww0 krb5kdc[31585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:51:54.763329 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:32805
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:32805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:55.131001 31617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:55.131345 31617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:55.131532 31617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:55.139613 31617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:55.139770 31617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:55.139834 31617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:55.139926 31617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:55.139991 31617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:55.150741 31617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:32805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:55.152208 31617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:55.153662 31617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:55.166676 31628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:55.166764 31631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:55.166745 31629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:55.167719 31617 server_base.cc:1034] running on GCE node
I20260809 22:51:55.168596 31617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:55.170169 31617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:55.171423 31617 hybrid_clock.cc:648] HybridClock initialized: now 1786315915171362 us; error 51 us; skew 500 ppm
I20260809 22:51:55.171882 31617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:55 dist-test-slave-pww0 krb5kdc[31585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315915, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:55.179442 31617 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:51:55.181375 31617 webserver.cc:460] Webserver started at http://127.24.136.62:38065/ using document root <none> and password file <none>
I20260809 22:51:55.182149 31617 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:55.182281 31617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:55.182611 31617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:55.185949 31617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "44f1e2d1501649fa9986ee9563f10b11"
format_stamp: "Formatted at 2026-08-09 22:51:55 on dist-test-slave-pww0"
server_key: "b1b44eac9183912bb6e5b7ce5cb4b91b"
server_key_iv: "67307ef2f93a88f24bc5a917f1fb6dca"
server_key_version: "encryptionkey@0"
I20260809 22:51:55.186803 31617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "44f1e2d1501649fa9986ee9563f10b11"
format_stamp: "Formatted at 2026-08-09 22:51:55 on dist-test-slave-pww0"
server_key: "b1b44eac9183912bb6e5b7ce5cb4b91b"
server_key_iv: "67307ef2f93a88f24bc5a917f1fb6dca"
server_key_version: "encryptionkey@0"
I20260809 22:51:55.192647 31617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 22:51:55.196652 31641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:55.198208 31617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:51:55.198385 31617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "44f1e2d1501649fa9986ee9563f10b11"
format_stamp: "Formatted at 2026-08-09 22:51:55 on dist-test-slave-pww0"
server_key: "b1b44eac9183912bb6e5b7ce5cb4b91b"
server_key_iv: "67307ef2f93a88f24bc5a917f1fb6dca"
server_key_version: "encryptionkey@0"
I20260809 22:51:55.198593 31617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:55.205005 31617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:55.209862 31617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:55.210175 31617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:55.229282 31617 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:32805
I20260809 22:51:55.229367 31730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:32805 every 8 connection(s)
I20260809 22:51:55.230844 31617 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:55.234968 31731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:55.238024 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31617
I20260809 22:51:55.238216 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:55.238795 25120 external_mini_cluster.cc:1428] Setting key 9b9e6486bba9bb019ccf9de4769e9331
I20260809 22:51:55.247715 31734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:55.241006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57575 (local address 127.24.136.62:32805)
0809 22:51:55.241916 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:55.241930 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:55.241978 (+ 48us) server_negotiation.cc:407] Connection header received
0809 22:51:55.243784 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:55.243815 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:55.244299 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:55.244852 (+ 553us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:55.245971 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:55.245977 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:55.245992 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:55.246008 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:55.246086 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:55.246107 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:55.246303 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:55.246407 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 22:51:55.246617 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":370,"threads_started":1}
I20260809 22:51:55.248525 31731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11: Bootstrap starting.
I20260809 22:51:55.252096 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:55.252445 31731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:55.253798 31731 log.cc:824] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:55.257640 31731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11: No bootstrap required, opened a new log
I20260809 22:51:55.260128 31734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:55.257405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38605 (local address 127.24.136.62:32805)
0809 22:51:55.257603 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:55.257609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:55.257937 (+ 328us) server_negotiation.cc:407] Connection header received
0809 22:51:55.258110 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:55.258116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:55.258199 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:55.258357 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:55.259310 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:55.259315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:55.259319 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:51:55.259339 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:55.259391 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:51:55.259417 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:55.259608 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:55.259751 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 22:51:55.259836 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:51:55.266494 31731 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f1e2d1501649fa9986ee9563f10b11" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32805 } }
I20260809 22:51:55.266925 31731 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:51:55.267031 31731 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f1e2d1501649fa9986ee9563f10b11, State: Initialized, Role: FOLLOWER
I20260809 22:51:55.267702 31731 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f1e2d1501649fa9986ee9563f10b11" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32805 } }
I20260809 22:51:55.267940 31731 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:51:55.268066 31731 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:51:55.268221 31731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:55.270561 31731 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f1e2d1501649fa9986ee9563f10b11" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32805 } }
I20260809 22:51:55.271188 31731 leader_election.cc:302] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44f1e2d1501649fa9986ee9563f10b11; no voters:
I20260809 22:51:55.271798 31731 leader_election.cc:288] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:51:55.271951 31747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:55.272614 31747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [term 1 LEADER]: Becoming Leader. State: Replica: 44f1e2d1501649fa9986ee9563f10b11, State: Running, Role: LEADER
I20260809 22:51:55.273255 31747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f1e2d1501649fa9986ee9563f10b11" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32805 } }
I20260809 22:51:55.274262 31731 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:55.276257 31753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f1e2d1501649fa9986ee9563f10b11. Latest consensus state: current_term: 1 leader_uuid: "44f1e2d1501649fa9986ee9563f10b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f1e2d1501649fa9986ee9563f10b11" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32805 } } }
I20260809 22:51:55.276770 31753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:55.277503 31752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44f1e2d1501649fa9986ee9563f10b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f1e2d1501649fa9986ee9563f10b11" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 32805 } } }
I20260809 22:51:55.278053 31752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:55.284155 31765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:55.289186 31765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:55.298666 31765 catalog_manager.cc:1347] Generated new cluster ID: 77ff786dfbb04a048c62cf40ca53f0c2
I20260809 22:51:55.298873 31765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:55.318696 31765 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:55.319171 31765 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:55.326452 31765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44f1e2d1501649fa9986ee9563f10b11: Generated new TSK 0
I20260809 22:51:55.327356 31765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:55.335268 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31617
W20260809 22:51:55.336565 31722 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57575
W20260809 22:51:55.337384 25120 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31617: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:51:55.337534 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31617
2026-08-09T22:51:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2229 ms)
[----------] 18 tests from AuthTokenIssuingTest (40539 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:51:55 dist-test-slave-pww0 krb5kdc[31788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:51:55 dist-test-slave-pww0 krb5kdc[31788](info): set up 2 sockets
Aug 09 22:51:55 dist-test-slave-pww0 krb5kdc[31788](info): commencing operation
krb5kdc: starting...
W20260809 22:51:56.648084 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:51:56 dist-test-slave-pww0 krb5kdc[31788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:51:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:51:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:51:56.994522 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45469
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45469,127.24.136.61:32787,127.24.136.60:38229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:51:57.362740 31810 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:51:57.363137 31810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:57.363240 31810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:57.363360 31810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:57.371032 31810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:57.371155 31810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:57.371241 31810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:57.371311 31810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:57.371402 31810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:57.381803 31810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45469,127.24.136.61:32787,127.24.136.60:38229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:45469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:57.383220 31810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:57.384608 31810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:57.396644 31817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:57.396646 31819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:57.396610 31816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:57.397784 31810 server_base.cc:1034] running on GCE node
I20260809 22:51:57.398699 31810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:57.400326 31810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:57.401582 31810 hybrid_clock.cc:648] HybridClock initialized: now 1786315917401495 us; error 54 us; skew 500 ppm
I20260809 22:51:57.401947 31810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:57 dist-test-slave-pww0 krb5kdc[31788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315917, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:57.405262 31810 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:51:57.406847 31810 webserver.cc:460] Webserver started at http://127.24.136.62:34187/ using document root <none> and password file <none>
I20260809 22:51:57.407624 31810 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:57.407742 31810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:57.408073 31810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:57.411267 31810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2"
format_stamp: "Formatted at 2026-08-09 22:51:57 on dist-test-slave-pww0"
server_key: "4c1f30e0d88e159105cbe0d21fc1b500"
server_key_iv: "c586773c0c7f2f5ccafeeb802625442a"
server_key_version: "encryptionkey@0"
I20260809 22:51:57.412215 31810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2"
format_stamp: "Formatted at 2026-08-09 22:51:57 on dist-test-slave-pww0"
server_key: "4c1f30e0d88e159105cbe0d21fc1b500"
server_key_iv: "c586773c0c7f2f5ccafeeb802625442a"
server_key_version: "encryptionkey@0"
I20260809 22:51:57.417764 31810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:51:57.421918 31832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:57.423417 31810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:51:57.423655 31810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2"
format_stamp: "Formatted at 2026-08-09 22:51:57 on dist-test-slave-pww0"
server_key: "4c1f30e0d88e159105cbe0d21fc1b500"
server_key_iv: "c586773c0c7f2f5ccafeeb802625442a"
server_key_version: "encryptionkey@0"
I20260809 22:51:57.423882 31810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:57.467604 31810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:57.472472 31810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:57.472774 31810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:57.492566 31810 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:45469
I20260809 22:51:57.492539 31917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:45469 every 8 connection(s)
I20260809 22:51:57.494120 31810 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:51:57.498238 31919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:57.501009 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31810
I20260809 22:51:57.501252 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:51:57.502120 25120 external_mini_cluster.cc:1428] Setting key 66351acaf2a43fbb2fe1caf835eb9f2a
I20260809 22:51:57.507043 31919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:51:57 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315917, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:57.531934 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.510919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.24.136.62:45469)
0809 22:51:57.512687 (+ 1768us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:57.512696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:57.512752 (+ 56us) server_negotiation.cc:407] Connection header received
0809 22:51:57.514371 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:57.514377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:57.514648 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:57.514843 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:57.516296 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:57.517646 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:57.519405 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:57.520059 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:57.520205 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:57.524366 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:57.524377 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:57.524391 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:57.524424 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:57.527360 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:57.528261 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:57.528269 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:57.528271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:57.528442 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:57.528926 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:57.528930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:57.528932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:57.529535 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:57.529713 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:57.530429 (+ 716us) server_negotiation.cc:299] Negotiation successful
0809 22:51:57.530742 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1535,"thread_start_us":1292,"threads_started":1}
I20260809 22:51:57.531960 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.510919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45469 (local address 127.0.0.1:38598)
0809 22:51:57.511799 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.511858 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:57.513934 (+ 2076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:57.514897 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:57.514903 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:57.515711 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:57.516070 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:57.516100 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:57.517852 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:57.517859 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:57.519232 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:57.519249 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:57.520312 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:57.520323 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:57.520492 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:57.521234 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:57.521264 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:57.524094 (+ 2830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:57.527510 (+ 3416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:57.527523 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:57.527538 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:57.528098 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:57.528558 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:57.528563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:57.528568 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:57.528779 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:57.529699 (+ 920us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:57.529708 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:57.530008 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 22:51:57.530296 (+ 288us) client_negotiation.cc:240] Negotiation successful
0809 22:51:57.530941 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":111,"thread_start_us":356,"threads_started":1}
I20260809 22:51:57.534956 31919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:51:57.536298 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.535815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43898)
0809 22:51:57.536021 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.536170 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 22:51:57.536628 31833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.24.136.61@KRBTEST.COM; defaulting to no policy
I20260809 22:51:57.538355 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.537897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43908)
0809 22:51:57.538082 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.538194 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:51:57.538589 31919 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32787: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab.
Entry for principal kudu/127.24.136.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab.
I20260809 22:51:57.578645 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.61:32787
--webserver_interface=127.24.136.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab
--principal=kudu/127.24.136.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45469,127.24.136.61:32787,127.24.136.60:38229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:51:57.587960 31919 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } attempt: 1
I20260809 22:51:57.589334 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.588796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43918)
0809 22:51:57.589008 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.589161 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 22:51:57.590549 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.590083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43922)
0809 22:51:57.590299 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.590383 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260809 22:51:57.590847 31919 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32787: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
I20260809 22:51:57.659162 31919 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } attempt: 2
I20260809 22:51:57.660650 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.660089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43938)
0809 22:51:57.660388 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.660491 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 22:51:57.661831 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.661432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43940)
0809 22:51:57.661587 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.661685 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:51:57.662055 31919 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32787: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
I20260809 22:51:57.753343 31919 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } attempt: 3
I20260809 22:51:57.754503 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.754050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43950)
0809 22:51:57.754267 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.754362 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 22:51:57.755750 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.755302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43954)
0809 22:51:57.755435 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.755577 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 22:51:57.755965 31919 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32787: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
I20260809 22:51:57.899228 31919 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } attempt: 4
I20260809 22:51:57.900601 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.900012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43958)
0809 22:51:57.900355 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.900453 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260809 22:51:57.901749 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:57.901348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43960)
0809 22:51:57.901521 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:57.901617 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 22:51:57.901966 31919 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32787: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32787: connect: Connection refused (error 111)
W20260809 22:51:57.946590 31931 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:51:57.946964 31931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:57.947041 31931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:57.947126 31931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:57.954715 31931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:57.954831 31931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:57.954891 31931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:57.954942 31931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:57.955005 31931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:57.965333 31931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45469,127.24.136.61:32787,127.24.136.60:38229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab
--principal=kudu/127.24.136.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.61:32787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.136.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:57.966701 31931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:57.968138 31931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:57.981196 31941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:57.981352 31936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:57.981577 31944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:57.982298 31931 server_base.cc:1034] running on GCE node
I20260809 22:51:57.983259 31931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:57.984925 31931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:57.986195 31931 hybrid_clock.cc:648] HybridClock initialized: now 1786315917986111 us; error 50 us; skew 500 ppm
I20260809 22:51:57.986647 31931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:57 dist-test-slave-pww0 krb5kdc[31788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315917, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:57.990471 31931 init.cc:377] Logged in from keytab as kudu/127.24.136.61@KRBTEST.COM (short username kudu)
I20260809 22:51:57.992254 31931 webserver.cc:460] Webserver started at http://127.24.136.61:42199/ using document root <none> and password file <none>
I20260809 22:51:57.993006 31931 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:57.993129 31931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:57.993458 31931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:57.996780 31931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/data/instance:
uuid: "f3fb2be3af1b41b685a380cafaa8ac99"
format_stamp: "Formatted at 2026-08-09 22:51:57 on dist-test-slave-pww0"
server_key: "0c09ea2bd9337ef2159ed830dc0ee23e"
server_key_iv: "4c3ea3918c9c77bcb281969746e7d289"
server_key_version: "encryptionkey@0"
I20260809 22:51:57.997634 31931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/wal/instance:
uuid: "f3fb2be3af1b41b685a380cafaa8ac99"
format_stamp: "Formatted at 2026-08-09 22:51:57 on dist-test-slave-pww0"
server_key: "0c09ea2bd9337ef2159ed830dc0ee23e"
server_key_iv: "4c3ea3918c9c77bcb281969746e7d289"
server_key_version: "encryptionkey@0"
I20260809 22:51:58.003733 31931 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 22:51:58.007922 31953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:58.009532 31931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:51:58.009754 31931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/wal
uuid: "f3fb2be3af1b41b685a380cafaa8ac99"
format_stamp: "Formatted at 2026-08-09 22:51:57 on dist-test-slave-pww0"
server_key: "0c09ea2bd9337ef2159ed830dc0ee23e"
server_key_iv: "4c3ea3918c9c77bcb281969746e7d289"
server_key_version: "encryptionkey@0"
I20260809 22:51:58.009994 31931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:58.044708 31931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:58.049779 31931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:58.050072 31931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:58.068372 31931 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.61:32787
I20260809 22:51:58.068406 32031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.61:32787 every 8 connection(s)
I20260809 22:51:58.069802 31931 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
I20260809 22:51:58.073691 32033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:58.074577 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 31931
I20260809 22:51:58.074772 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-1/wal/instance
I20260809 22:51:58.075217 25120 external_mini_cluster.cc:1428] Setting key 2623c001f31954d83fb4f21af624c814
I20260809 22:51:58.080734 32033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315917, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:58.103603 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.083587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.24.136.62:45469)
0809 22:51:58.084099 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.084103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.085239 (+ 1136us) server_negotiation.cc:407] Connection header received
0809 22:51:58.087234 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.087240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.087339 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.087441 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.089159 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.089946 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.091697 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.092334 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.092470 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.096346 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.096353 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.096358 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.096444 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.100006 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.100934 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.100941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.100946 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.101106 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.101568 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.101573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.101575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.101952 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.102102 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.102926 (+ 824us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.103166 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":277,"threads_started":1}
I20260809 22:51:58.104691 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.084269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45469 (local address 127.0.0.1:38608)
0809 22:51:58.085064 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.085103 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:58.086914 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:58.087750 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:58.087759 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:58.088320 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:58.088931 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.088949 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.090121 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.090126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.091528 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.091543 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.092612 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.092618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.092877 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.093602 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:58.093642 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:58.096035 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:58.100201 (+ 4166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.100210 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.100227 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.100760 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.101228 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.101232 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.101234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.101444 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.102103 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:58.102111 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:58.102502 (+ 391us) client_negotiation.cc:769] Sending connection context
0809 22:51:58.102757 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 22:51:58.103240 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":341,"threads_started":1}
I20260809 22:51:58.106717 32033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.136.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.60@KRBTEST.COM" created.
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315917, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:51:58.122699 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.107671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43968)
0809 22:51:58.107843 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.107863 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:58.107979 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:58.109148 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:58.109152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:58.109524 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:58.109681 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.109689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.111229 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.111235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.112129 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.112142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.112960 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.112965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.113170 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.113830 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:58.113852 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:58.116469 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:58.120045 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.120051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.120055 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.120473 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.120891 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.120896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.120900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.121044 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.121730 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:58.121735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:58.121877 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 22:51:58.121991 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 22:51:58.122306 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 22:51:58.122815 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.107952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.24.136.61:32787)
0809 22:51:58.108675 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.108681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.108707 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:51:58.108754 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.108759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.108962 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.109106 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.109839 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.111103 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.112260 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.112819 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.112933 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.116771 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.116787 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.116804 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.116847 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.119886 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.120634 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.120641 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.120644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.120778 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.121164 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.121168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.121170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.121598 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.121804 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.122159 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.122435 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":564,"thread_start_us":416,"threads_started":1}
I20260809 22:51:58.125921 32033 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:51:58.127166 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.126653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38620)
0809 22:51:58.126837 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.126981 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 22:51:58.127424 31956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
I20260809 22:51:58.129079 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.128722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38624)
0809 22:51:58.128871 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.128951 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:51:58.129333 32033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38229: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab.
Entry for principal kudu/127.24.136.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab.
I20260809 22:51:58.150643 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.60:38229
--webserver_interface=127.24.136.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab
--principal=kudu/127.24.136.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:37907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:45469,127.24.136.61:32787,127.24.136.60:38229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:51:58.178666 32033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } attempt: 1
I20260809 22:51:58.179908 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.179374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38638)
0809 22:51:58.179635 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.179750 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 22:51:58.181074 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.180672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38640)
0809 22:51:58.180864 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.180954 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 22:51:58.181306 32033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38229: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
I20260809 22:51:58.201210 31919 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } attempt: 5
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315917, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:51:58.216883 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.201993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43972)
0809 22:51:58.202227 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.202249 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:58.202402 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:58.202789 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:58.202794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:58.203208 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:58.203402 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.203411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.204919 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.204923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.205801 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.205815 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.206877 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.206882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.207049 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.207738 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:58.207757 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:58.209985 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:58.214231 (+ 4246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.214237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.214241 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.214722 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.215203 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.215207 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.215209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.215382 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.216120 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:58.216124 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:58.216272 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 22:51:58.216391 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 22:51:58.216638 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 22:51:58.217073 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.202052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.24.136.61:32787)
0809 22:51:58.202243 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.202250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.202332 (+ 82us) server_negotiation.cc:407] Connection header received
0809 22:51:58.202543 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.202547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.202636 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.202727 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.203613 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.204733 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.205981 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.206644 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.206822 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.210200 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.210207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.210212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.210255 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.214034 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.214866 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.214873 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.214877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.215059 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.215524 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.215529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.215533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.215988 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.216148 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.216531 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.216776 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 22:51:58.218268 31919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:51:58.219338 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.218942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38644)
0809 22:51:58.219118 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.219195 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 22:51:58.220254 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.219913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38650)
0809 22:51:58.220051 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.220127 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 22:51:58.220438 31919 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38229: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
I20260809 22:51:58.249528 32033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } attempt: 2
I20260809 22:51:58.250766 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.250288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38656)
0809 22:51:58.250497 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.250596 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:51:58.251798 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.251430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38662)
0809 22:51:58.251592 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.251672 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 22:51:58.252012 32033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38229: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
I20260809 22:51:58.271674 31919 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } attempt: 1
I20260809 22:51:58.272987 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.272440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38676)
0809 22:51:58.272692 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.272795 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260809 22:51:58.274065 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.273694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38684)
0809 22:51:58.273856 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.273936 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:51:58.274289 31919 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38229: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
I20260809 22:51:58.342541 31919 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } attempt: 2
I20260809 22:51:58.343222 32033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } attempt: 3
I20260809 22:51:58.343922 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.343276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38690)
0809 22:51:58.343580 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.343702 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 22:51:58.344291 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.343801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38704)
0809 22:51:58.344000 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.344100 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 22:51:58.345006 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.344597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38706)
0809 22:51:58.344772 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.344867 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:51:58.345228 31919 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38229: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
I20260809 22:51:58.345674 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.345251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38708)
0809 22:51:58.345470 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.345559 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260809 22:51:58.345863 32033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38229: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
I20260809 22:51:58.451532 31919 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } attempt: 3
I20260809 22:51:58.452803 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.452296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38716)
0809 22:51:58.452514 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.452613 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 22:51:58.453888 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.453457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38720)
0809 22:51:58.453692 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.453777 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260809 22:51:58.454099 31919 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38229: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
I20260809 22:51:58.489101 32033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } attempt: 4
I20260809 22:51:58.490437 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.489884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38724)
0809 22:51:58.490157 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.490253 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 22:51:58.491420 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.491081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38730)
0809 22:51:58.491233 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.491306 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 22:51:58.491705 32033 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38229: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38229: connect: Connection refused (error 111)
W20260809 22:51:58.518728 32044 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:51:58.519084 32044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:51:58.519177 32044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:51:58.519294 32044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:51:58.526737 32044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:51:58.526861 32044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:51:58.526957 32044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:51:58.527029 32044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:51:58.527123 32044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:51:58.537685 32044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:37907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:45469,127.24.136.61:32787,127.24.136.60:38229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab
--principal=kudu/127.24.136.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.60:38229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.136.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:51:58.539096 32044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:51:58.540508 32044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:51:58.552618 32062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:58.552767 32060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:51:58.552745 32066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:51:58.553689 32044 server_base.cc:1034] running on GCE node
I20260809 22:51:58.554510 32044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:51:58.556046 32044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:51:58.557328 32044 hybrid_clock.cc:648] HybridClock initialized: now 1786315918557255 us; error 45 us; skew 500 ppm
I20260809 22:51:58.557706 32044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315918, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:51:58.560967 32044 init.cc:377] Logged in from keytab as kudu/127.24.136.60@KRBTEST.COM (short username kudu)
I20260809 22:51:58.562422 32044 webserver.cc:460] Webserver started at http://127.24.136.60:33397/ using document root <none> and password file <none>
I20260809 22:51:58.563130 32044 fs_manager.cc:359] Metadata directory not provided
I20260809 22:51:58.563235 32044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:51:58.563604 32044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:51:58.566736 32044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/data/instance:
uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf"
format_stamp: "Formatted at 2026-08-09 22:51:58 on dist-test-slave-pww0"
server_key: "bd76b9238982d2b0d41d63ffeed29130"
server_key_iv: "9b2dc3fb46d11a2c36104c5eef0f2e8c"
server_key_version: "encryptionkey@0"
I20260809 22:51:58.567718 32044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/wal/instance:
uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf"
format_stamp: "Formatted at 2026-08-09 22:51:58 on dist-test-slave-pww0"
server_key: "bd76b9238982d2b0d41d63ffeed29130"
server_key_iv: "9b2dc3fb46d11a2c36104c5eef0f2e8c"
server_key_version: "encryptionkey@0"
I20260809 22:51:58.572983 32044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:51:58.576846 32077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:51:58.578277 32044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:51:58.578464 32044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/wal
uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf"
format_stamp: "Formatted at 2026-08-09 22:51:58 on dist-test-slave-pww0"
server_key: "bd76b9238982d2b0d41d63ffeed29130"
server_key_iv: "9b2dc3fb46d11a2c36104c5eef0f2e8c"
server_key_version: "encryptionkey@0"
I20260809 22:51:58.578722 32044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:51:58.591717 32044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:51:58.596103 32044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:51:58.596377 32044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:51:58.614338 32044 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.60:38229
I20260809 22:51:58.614362 32164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.60:38229 every 8 connection(s)
I20260809 22:51:58.615926 32044 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
I20260809 22:51:58.619843 32165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:51:58.625085 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 32044
I20260809 22:51:58.625274 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315533505708-25120-0/minicluster-data/master-2/wal/instance
I20260809 22:51:58.625775 25120 external_mini_cluster.cc:1428] Setting key 975c9309a3a8f89afe3749d5c4f8bb1a
I20260809 22:51:58.626756 32165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:51:58.631319 31919 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } attempt: 4
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315918, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315917, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
I20260809 22:51:58.649186 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.631868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38756)
0809 22:51:58.632217 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.632236 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:58.632334 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:58.633301 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:58.633306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:58.633833 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:58.634044 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.634058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.635564 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.635569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.636955 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.636969 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.637875 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.637881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.638093 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.638647 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:58.638695 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:58.641838 (+ 3143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:58.646256 (+ 4418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.646278 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.646281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.646810 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.647199 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.647205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.647208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.647370 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.648276 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:58.648281 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:58.648443 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 22:51:58.648565 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 22:51:58.648835 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260809 22:51:58.650074 32171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.629856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.24.136.62:45469)
0809 22:51:58.631319 (+ 1463us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.631325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.631347 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:51:58.631429 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.631433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.631553 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.631682 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.633730 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.634696 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.636887 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.637513 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.637628 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.641751 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.641757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.641761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.641861 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.644959 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.646027 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.646033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.646037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.646177 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.646757 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.646762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.646765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.647233 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.647385 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.649411 (+ 2026us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.649641 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1341,"spinlock_wait_cycles":20736,"thread_start_us":1211,"threads_started":1}
I20260809 22:51:58.650187 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.627717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38742 (local address 127.24.136.60:38229)
0809 22:51:58.628577 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.628592 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.628643 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:51:58.631005 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.631034 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.631344 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.631687 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.633378 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.634982 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.636530 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.637291 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.637502 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.642742 (+ 5240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.642750 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.642754 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.642786 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.646380 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.647244 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.647248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.647251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.647390 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.647812 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.647818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.647821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.648114 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.648432 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.648757 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.649010 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":593,"thread_start_us":354,"threads_started":1}
I20260809 22:51:58.650416 32173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.632335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38756 (local address 127.24.136.60:38229)
0809 22:51:58.632897 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.632902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.632924 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:51:58.632968 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.632977 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.633112 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.633236 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.634243 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.635399 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.637083 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.637664 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.637847 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.642099 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.642110 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.642128 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.642165 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.646122 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.646936 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.646946 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.646949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.647090 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.647542 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.647547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.647550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.648093 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.649636 (+ 1543us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.649760 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.649962 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":438,"spinlock_wait_cycles":20864,"thread_start_us":296,"threads_started":1}
I20260809 22:51:58.650707 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.629797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:45469 (local address 127.0.0.1:38622)
0809 22:51:58.630527 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.630582 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:58.631007 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:58.631925 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:58.631932 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:58.632894 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:58.633555 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.633584 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.634866 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.634872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.636694 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.636710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.637876 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.637884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.638093 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.638822 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:58.638853 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:58.641451 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:58.645102 (+ 3651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.645116 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.645136 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.645893 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.646323 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.646328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.646337 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.646617 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.647367 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:58.647375 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:58.647918 (+ 543us) client_negotiation.cc:769] Sending connection context
0809 22:51:58.648232 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 22:51:58.648806 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":68,"spinlock_wait_cycles":36736,"thread_start_us":280,"threads_started":1}
I20260809 22:51:58.653452 32165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:51:58.657693 31919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2: Bootstrap starting.
I20260809 22:51:58.661755 31919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2: Neither blocks nor log segments found. Creating new log.
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315918, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:51:58.663218 31919 log.cc:824] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:51:58.666728 31919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2: No bootstrap required, opened a new log
I20260809 22:51:58.669497 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.654221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32787 (local address 127.0.0.1:43984)
0809 22:51:58.654428 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.654446 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:58.654550 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:58.655142 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:58.655147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:58.655638 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:58.655854 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.655866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.657204 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.657209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.658520 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.658536 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.659527 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.659531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.659717 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.660291 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:58.660309 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:58.662993 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:58.666752 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.666759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.666764 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.667348 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.667834 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.667840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.667844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.667991 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.668506 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:58.668511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:58.668669 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 22:51:58.668810 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:51:58.669132 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 22:51:58.669533 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.654632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.24.136.61:32787)
0809 22:51:58.654824 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.654830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.654848 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:51:58.654907 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.654911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.654985 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.655080 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.656047 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.657037 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.658682 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.659340 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.659520 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.663245 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.663252 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.663257 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.663306 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.666593 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.667523 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.667530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.667533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.667705 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.668103 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.668108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.668110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.668388 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.668529 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.668923 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.669145 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:51:58.670817 32165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:51:58.672590 32171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.653704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38626 (local address 127.24.136.62:45469)
0809 22:51:58.653885 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.653891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.654020 (+ 129us) server_negotiation.cc:407] Connection header received
0809 22:51:58.654191 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.654196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.654310 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.654434 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.656158 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.656889 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.659727 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.660358 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.660484 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.665217 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.665224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.665229 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.665269 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.668899 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.669903 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.669909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.669913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.670082 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.670592 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.670598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.670601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.671020 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.671296 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.672009 (+ 713us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.672260 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315918, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
I20260809 22:51:58.678028 31919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } }
I20260809 22:51:58.678399 31919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 683369ce24ba42f98f5b2a0eb0d3e3e2, State: Initialized, Role: FOLLOWER
I20260809 22:51:58.678906 31919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } }
I20260809 22:51:58.680160 32177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.680713 32177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:51:58.681591 31919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:51:58.685891 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.671500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38762)
0809 22:51:58.671661 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.671680 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:58.671778 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:58.672108 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:58.672112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:58.672503 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:58.672667 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.672679 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.673693 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.673696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.674801 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.674822 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.675783 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.675789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.676009 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.676597 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:58.676615 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:58.678969 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:58.682712 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.682717 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.682721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.683219 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.683716 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.683722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.683727 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.683930 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.684664 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:58.684670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:58.684871 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 22:51:58.685044 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 22:51:58.685415 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 22:51:58.685971 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.671739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.24.136.60:38229)
0809 22:51:58.671884 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.671889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.671904 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:51:58.671941 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.671944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.671992 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.672068 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.672813 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.673582 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.674964 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.675623 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.675759 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.679153 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.679157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.679160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.679190 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.682562 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.683370 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.683375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.683379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.683563 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.684053 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.684059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.684063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.684538 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.684681 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.685173 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.685611 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":167,"server-negotiator.queue_time_us":53}
I20260809 22:51:58.690912 32165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf: Bootstrap starting.
I20260809 22:51:58.694653 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.675330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.24.136.61:32787)
0809 22:51:58.675519 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.675525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.675608 (+ 83us) server_negotiation.cc:407] Connection header received
0809 22:51:58.675801 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.675806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.675888 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.675985 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.677611 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.678710 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.680431 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.681064 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.681208 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.687598 (+ 6390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.687608 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.687613 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.687665 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.691404 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.692315 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.692320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.692325 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.692519 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.692973 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.692978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.692981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.693363 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.693657 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.693977 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.694255 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:51:58.695087 32165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:58.696663 32165 log.cc:824] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:58.700742 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:51:58.704823 31875 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38626
I20260809 22:51:58.705269 31875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 22:51:58.705487 31875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:58.706647 32165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf: No bootstrap required, opened a new log
I20260809 22:51:58.709650 31875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } }
I20260809 22:51:58.713330 31875 leader_election.cc:288] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [CANDIDATE]: Term 1 election: Requested vote from peers f3fb2be3af1b41b685a380cafaa8ac99 (127.24.136.61:32787), e3ffb88c9f44421c8e5e7e2dcf9aafdf (127.24.136.60:38229)
I20260809 22:51:58.702373 31810 master_runner.cc:186] Error getting master registration for 127.24.136.61:32787: OK, instance_id { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" instance_seqno: 1786315918063055 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:51:58.714929 31810 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:51:58.720158 32165 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } }
I20260809 22:51:58.720626 32165 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ffb88c9f44421c8e5e7e2dcf9aafdf, State: Initialized, Role: FOLLOWER
I20260809 22:51:58.721190 32165 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } }
W20260809 22:51:58.725947 31842 tablet.cc:2369] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:51:58.728922 32192 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:51:58.729055 32192 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:51:58.728703 32204 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.729161 32204 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:51:58.731196 32165 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:58.731304 32121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf"
I20260809 22:51:58.731917 32121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:58.740231 32121 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 683369ce24ba42f98f5b2a0eb0d3e3e2 in term 1.
I20260809 22:51:58.741084 32171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.720286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.24.136.62:45469)
0809 22:51:58.721187 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.721194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.721304 (+ 110us) server_negotiation.cc:407] Connection header received
0809 22:51:58.721652 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.721658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.721765 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.721841 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.724372 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.725697 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.728011 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.728699 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.728804 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.732707 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.732715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.732721 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.732782 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.736984 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.738071 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.738077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.738081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.738277 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.738839 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.738844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.738847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.739296 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.739670 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.740356 (+ 686us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.740657 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753}
I20260809 22:51:58.742694 31834 leader_election.cc:302] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 683369ce24ba42f98f5b2a0eb0d3e3e2, e3ffb88c9f44421c8e5e7e2dcf9aafdf; no voters:
I20260809 22:51:58.743304 32177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:51:58.744659 32177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [term 1 LEADER]: Becoming Leader. State: Replica: 683369ce24ba42f98f5b2a0eb0d3e3e2, State: Running, Role: LEADER
I20260809 22:51:58.745325 32177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } }
I20260809 22:51:58.752851 32214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 683369ce24ba42f98f5b2a0eb0d3e3e2. Latest consensus state: current_term: 1 leader_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.753357 32214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:58.754560 32218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:51:58.757182 32218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:51:58.767406 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.721543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44016 (local address 127.24.136.61:32787)
0809 22:51:58.721727 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.721733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.742493 (+ 20760us) server_negotiation.cc:407] Connection header received
0809 22:51:58.742799 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.742807 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.742901 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.742973 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.752088 (+ 9115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.753204 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.755105 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.755782 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.755894 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.759734 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.759742 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.759746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.759793 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.763730 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.764748 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.764755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.764759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.764959 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.765509 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.765515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.765518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.765917 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.766163 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.766772 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.767029 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:51:58.776185 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.722519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.24.136.60:38229)
0809 22:51:58.722712 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.722718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.756461 (+ 33743us) server_negotiation.cc:407] Connection header received
0809 22:51:58.758771 (+ 2310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.758779 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.758879 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.758958 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.761604 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.762691 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.764719 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.765428 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.765535 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.768736 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.768744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.768747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.768808 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.772543 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.773505 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.773512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.773516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.773733 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.774224 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.774230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.774233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.774686 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.774932 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.775574 (+ 642us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.775841 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:51:58.779714 32121 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [term 1 FOLLOWER]: Refusing update from remote peer 683369ce24ba42f98f5b2a0eb0d3e3e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:58.780939 32214 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:51:58.790925 32033 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } attempt: 5
Aug 09 22:51:58 dist-test-slave-pww0 krb5kdc[31788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315917, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
I20260809 22:51:58.818719 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.792378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38229 (local address 127.0.0.1:38776)
0809 22:51:58.792628 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 22:51:58.792649 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:51:58.792784 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:51:58.793357 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:51:58.793362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:51:58.793932 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:51:58.794144 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.794156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.797605 (+ 3449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.797612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.799091 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:51:58.799110 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.800235 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.800242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:51:58.800516 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.801219 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:51:58.801242 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:51:58.804245 (+ 3003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:51:58.814206 (+ 9961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.814215 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.814220 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.814903 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.816083 (+ 1180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:51:58.816089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:51:58.816093 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:51:58.816308 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:51:58.817542 (+ 1234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:51:58.817548 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:51:58.817744 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 22:51:58.817905 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 22:51:58.818272 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260809 22:51:58.820895 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:58.791899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38776 (local address 127.24.136.60:38229)
0809 22:51:58.792077 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:58.792083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:58.792901 (+ 818us) server_negotiation.cc:407] Connection header received
0809 22:51:58.792969 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:58.792975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:58.793081 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:58.793170 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:58.794306 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.795387 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.799259 (+ 3872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:58.800076 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:58.800212 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:58.804490 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:51:58.804500 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:51:58.804506 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:51:58.804557 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:51:58.813939 (+ 9382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.815238 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.815247 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.815251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.815856 (+ 605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:51:58.816515 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:51:58.816521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:51:58.816525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:51:58.817331 (+ 806us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:51:58.819626 (+ 2295us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:58.819839 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 22:51:58.820300 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:51:58.829873 32033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99: Bootstrap starting.
I20260809 22:51:58.834416 32033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99: Neither blocks nor log segments found. Creating new log.
I20260809 22:51:58.835968 32033 log.cc:824] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99: Log is configured to *not* fsync() on all Append() calls
I20260809 22:51:58.837786 32204 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 683369ce24ba42f98f5b2a0eb0d3e3e2. Latest consensus state: current_term: 1 leader_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.838145 32204 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:51:58.842589 32220 mvcc.cc:205] Tried to move back new op lower bound from 7316750003303505920 to 7316750003187965952. Current Snapshot: MvccSnapshot[applied={T|T < 7316750003303505920}]
I20260809 22:51:58.845816 32033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99: No bootstrap required, opened a new log
I20260809 22:51:58.858698 32177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.859084 32177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:58.859488 32218 catalog_manager.cc:1347] Generated new cluster ID: 83a8445608264a5aae1581b92955795f
I20260809 22:51:58.859603 32218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:51:58.863703 32204 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.863976 32204 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:51:58.871501 32033 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } }
I20260809 22:51:58.871984 32033 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3fb2be3af1b41b685a380cafaa8ac99, State: Initialized, Role: FOLLOWER
I20260809 22:51:58.872562 32033 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } }
I20260809 22:51:58.873718 32240 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.874207 32240 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:51:58.877203 32033 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:51:58.877616 31995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:51:58.881279 31996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f3fb2be3af1b41b685a380cafaa8ac99"
I20260809 22:51:58.882702 31995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [term 1 FOLLOWER]: Refusing update from remote peer 683369ce24ba42f98f5b2a0eb0d3e3e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:51:58.886411 32244 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:51:58.887194 31996 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 683369ce24ba42f98f5b2a0eb0d3e3e2 in term 1.
I20260809 22:51:58.894726 32218 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:51:58.897408 32218 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:51:58.903896 32214 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:51:58.894385 32044 master_runner.cc:186] Error getting master registration for 127.24.136.61:32787: OK, instance_id { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" instance_seqno: 1786315918063055 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:51:58.914773 32044 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:51:58.914705 32218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2: Generated new TSK 0
I20260809 22:51:58.916009 32218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:51:58.943590 32240 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 683369ce24ba42f98f5b2a0eb0d3e3e2. Latest consensus state: current_term: 1 leader_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.943987 32240 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:51:58.944092 32262 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.944458 32262 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:51:58.945662 32177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "683369ce24ba42f98f5b2a0eb0d3e3e2" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 45469 } } peers { permanent_uuid: "f3fb2be3af1b41b685a380cafaa8ac99" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32787 } } peers { permanent_uuid: "e3ffb88c9f44421c8e5e7e2dcf9aafdf" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38229 } } }
I20260809 22:51:58.946014 32177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 683369ce24ba42f98f5b2a0eb0d3e3e2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:51:58.966984 32267 catalog_manager.cc:1259] Loaded cluster ID: 83a8445608264a5aae1581b92955795f
I20260809 22:51:58.967180 32267 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99: loading cluster ID for follower catalog manager: success
I20260809 22:51:58.971221 32267 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99: acquiring CA information for follower catalog manager: success
I20260809 22:51:58.974098 32267 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f3fb2be3af1b41b685a380cafaa8ac99: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 22:51:59.034127 32040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.020416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.24.136.61:32787)
0809 22:51:59.020745 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.020752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.031983 (+ 11231us) server_negotiation.cc:407] Connection header received
0809 22:51:59.032519 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.032525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.032660 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.032786 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.033880 (+ 1094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44022: BlockingRecv error: recv got EOF from 127.0.0.1:44022 (error 108)
Metrics: {"server-negotiator.queue_time_us":171}
W20260809 22:51:59.036989 32171 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.020338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38644 (local address 127.24.136.62:45469)
0809 22:51:59.020559 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.020566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.034984 (+ 14418us) server_negotiation.cc:407] Connection header received
0809 22:51:59.035166 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.035171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.035263 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.035371 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.036711 (+ 1340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38644: BlockingRecv error: recv got EOF from 127.0.0.1:38644 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 22:51:59.045006 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.035731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.24.136.60:38229)
0809 22:51:59.035930 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.035936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.036080 (+ 144us) server_negotiation.cc:407] Connection header received
0809 22:51:59.036291 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.036298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.036390 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.036486 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.044705 (+ 8219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38780: BlockingRecv error: recv got EOF from 127.0.0.1:38780 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:51:59.058768 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.050466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44038 (local address 127.24.136.61:32787)
0809 22:51:59.050675 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.050681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.051235 (+ 554us) server_negotiation.cc:407] Connection header received
0809 22:51:59.051538 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.051545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.051779 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.051970 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.052473 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.053692 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.055820 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.056514 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.056672 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.057251 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.058005 (+ 754us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.058163 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.058290 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.058397 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 22:51:59.069801 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.067238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38790 (local address 127.24.136.60:38229)
0809 22:51:59.067481 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.067488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.067618 (+ 130us) server_negotiation.cc:407] Connection header received
0809 22:51:59.068215 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.068222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.068367 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.068447 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.069566 (+ 1119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38790: BlockingRecv error: recv got EOF from 127.0.0.1:38790 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:51:59.115048 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.103203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.24.136.61:32787)
0809 22:51:59.103439 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.103446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.108142 (+ 4696us) server_negotiation.cc:407] Connection header received
0809 22:51:59.108428 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.108435 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.108546 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.108659 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.109158 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.110353 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.112410 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.113169 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.113309 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.113906 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.114134 (+ 228us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.114244 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.114471 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.114572 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 22:51:59.126034 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.124109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38806 (local address 127.24.136.60:38229)
0809 22:51:59.124322 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.124328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.124350 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:51:59.124454 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.124460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.124575 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.124697 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.125762 (+ 1065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38806: BlockingRecv error: recv got EOF from 127.0.0.1:38806 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:51:59.161624 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.153882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.24.136.61:32787)
0809 22:51:59.154092 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.154099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.154685 (+ 586us) server_negotiation.cc:407] Connection header received
0809 22:51:59.154942 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.154950 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.155064 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.155175 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.155810 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.157083 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.159156 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.159932 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.160091 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.160634 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.160826 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.160937 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.161120 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.161233 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 22:51:59.177938 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.173700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.24.136.60:38229)
0809 22:51:59.173905 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.173912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.176172 (+ 2260us) server_negotiation.cc:407] Connection header received
0809 22:51:59.176422 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.176428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.176535 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.176656 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.177666 (+ 1010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38808: BlockingRecv error: recv got EOF from 127.0.0.1:38808 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:51:59.199509 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.191688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44060 (local address 127.24.136.61:32787)
0809 22:51:59.191966 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.191973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.192576 (+ 603us) server_negotiation.cc:407] Connection header received
0809 22:51:59.192841 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.192847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.192948 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.193058 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.193583 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.194766 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.197063 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.197794 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.197932 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.198604 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.198780 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.198892 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.199060 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.199169 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
W20260809 22:51:59.213768 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.208768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38818 (local address 127.24.136.60:38229)
0809 22:51:59.209041 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.209048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.212105 (+ 3057us) server_negotiation.cc:407] Connection header received
0809 22:51:59.212321 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.212327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.212449 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.212553 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.213529 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38818: BlockingRecv error: recv got EOF from 127.0.0.1:38818 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
I20260809 22:51:59.233425 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.225990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.24.136.61:32787)
0809 22:51:59.226191 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.226197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.226761 (+ 564us) server_negotiation.cc:407] Connection header received
0809 22:51:59.226963 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.226969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.227064 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.227159 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.227745 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.228904 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.230954 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.231807 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.231943 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.232518 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.232723 (+ 205us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.232811 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.232992 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.233087 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 22:51:59.251947 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.249266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.24.136.60:38229)
0809 22:51:59.249478 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.249486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.250155 (+ 669us) server_negotiation.cc:407] Connection header received
0809 22:51:59.250509 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.250515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.250629 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.250739 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.251719 (+ 980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38834: BlockingRecv error: recv got EOF from 127.0.0.1:38834 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:51:59.286798 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.280035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.24.136.61:32787)
0809 22:51:59.280268 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.280274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.280726 (+ 452us) server_negotiation.cc:407] Connection header received
0809 22:51:59.280900 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.280906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.281004 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.281170 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.281618 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.282909 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.284564 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.285239 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.285381 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.285843 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.286069 (+ 226us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.286171 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.286384 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.286474 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260809 22:51:59.295930 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.293922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38848 (local address 127.24.136.60:38229)
0809 22:51:59.294148 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.294155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.294443 (+ 288us) server_negotiation.cc:407] Connection header received
0809 22:51:59.294615 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.294621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.294720 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.294843 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.295645 (+ 802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38848: BlockingRecv error: recv got EOF from 127.0.0.1:38848 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:51:59.338891 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.332229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44084 (local address 127.24.136.61:32787)
0809 22:51:59.332579 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.332589 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.332902 (+ 313us) server_negotiation.cc:407] Connection header received
0809 22:51:59.333197 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.333202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.333296 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.333395 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.333954 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.334803 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.336900 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.337436 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.337550 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.338085 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.338209 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.338286 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.338468 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.338549 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260809 22:51:59.347903 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.345752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.24.136.60:38229)
0809 22:51:59.345977 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.345984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.346321 (+ 337us) server_negotiation.cc:407] Connection header received
0809 22:51:59.346497 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.346504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.346614 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.346746 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.347647 (+ 901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38856: BlockingRecv error: recv got EOF from 127.0.0.1:38856 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:51:59.425854 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.418791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44096 (local address 127.24.136.61:32787)
0809 22:51:59.418982 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.418989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.419704 (+ 715us) server_negotiation.cc:407] Connection header received
0809 22:51:59.419941 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.419946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.420038 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.420128 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.420649 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.421623 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.423655 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.424381 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.424530 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.425063 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.425231 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.425322 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.425459 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.425553 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 22:51:59.439126 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.436840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38868 (local address 127.24.136.60:38229)
0809 22:51:59.437029 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.437036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.437514 (+ 478us) server_negotiation.cc:407] Connection header received
0809 22:51:59.437726 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.437734 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.437860 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.437998 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.438910 (+ 912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38868: BlockingRecv error: recv got EOF from 127.0.0.1:38868 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:51:59.576459 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.571621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.24.136.61:32787)
0809 22:51:59.571886 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.571893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.572141 (+ 248us) server_negotiation.cc:407] Connection header received
0809 22:51:59.572327 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.572332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.572408 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.572491 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.572896 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.573606 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.574902 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.575308 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.575399 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.575787 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.575902 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.575965 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.576092 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.576164 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 22:51:59.584638 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.582564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.24.136.60:38229)
0809 22:51:59.582773 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.582779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.583258 (+ 479us) server_negotiation.cc:407] Connection header received
0809 22:51:59.583481 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.583487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.583574 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.583662 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.584466 (+ 804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38870: BlockingRecv error: recv got EOF from 127.0.0.1:38870 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 22:51:59.850503 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:51:59.845051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44122 (local address 127.24.136.61:32787)
0809 22:51:59.845337 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.845345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.845598 (+ 253us) server_negotiation.cc:407] Connection header received
0809 22:51:59.845793 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.845797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.845873 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.845957 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:51:59.846342 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.847080 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.848732 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:51:59.849209 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:51:59.849328 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:51:59.849788 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:51:59.849906 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:51:59.849971 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 22:51:59.850128 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 22:51:59.850204 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260809 22:51:59.859414 32169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:51:59.857613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38872 (local address 127.24.136.60:38229)
0809 22:51:59.857814 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:51:59.857819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:51:59.858097 (+ 278us) server_negotiation.cc:407] Connection header received
0809 22:51:59.858275 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:51:59.858280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:51:59.858361 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:51:59.858458 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:51:59.859189 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38872: BlockingRecv error: recv got EOF from 127.0.0.1:38872 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:51:59.891237 32244 catalog_manager.cc:1259] Loaded cluster ID: 83a8445608264a5aae1581b92955795f
I20260809 22:51:59.891363 32244 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf: loading cluster ID for follower catalog manager: success
I20260809 22:51:59.894248 32244 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf: acquiring CA information for follower catalog manager: success
I20260809 22:51:59.896406 32244 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3ffb88c9f44421c8e5e7e2dcf9aafdf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:52:00.380540 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:00.375060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44130 (local address 127.24.136.61:32787)
0809 22:52:00.375657 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:00.375663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:00.375756 (+ 93us) server_negotiation.cc:407] Connection header received
0809 22:52:00.375988 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:00.375992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:00.376064 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:00.376174 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:00.376617 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.377356 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.378770 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.379217 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.379337 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:00.379772 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:00.379884 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:00.379959 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:00.380130 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 22:52:00.380223 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":304,"threads_started":1}
I20260809 22:52:00.392915 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:00.386437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.24.136.60:38229)
0809 22:52:00.386960 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:00.386967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:00.387217 (+ 250us) server_negotiation.cc:407] Connection header received
0809 22:52:00.387366 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:00.387370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:00.387621 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:00.387745 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:00.388228 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.388962 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.390469 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.390940 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.391063 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:00.391687 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:00.392256 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:00.392418 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:00.392513 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 22:52:00.392610 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":252,"threads_started":1}
I20260809 22:52:00.407042 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:00.399236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.24.136.62:45469)
0809 22:52:00.399932 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:00.399940 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:00.400566 (+ 626us) server_negotiation.cc:407] Connection header received
0809 22:52:00.400927 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:00.400934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:00.401185 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:00.401394 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:00.401951 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.403009 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.404594 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.405214 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.405352 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:00.405995 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:00.406409 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:00.406507 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:00.406639 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 22:52:00.406737 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":475,"threads_started":1}
I20260809 22:52:00.408226 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:00.399306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44136 (local address 127.24.136.61:32787)
0809 22:52:00.399541 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:00.399548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:00.402037 (+ 2489us) server_negotiation.cc:407] Connection header received
0809 22:52:00.402159 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:00.402164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:00.402250 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:00.402338 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:00.402882 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.403998 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.405981 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.406546 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.406670 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:00.407391 (+ 721us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:00.407566 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:00.407630 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:00.407856 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 22:52:00.407943 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:52:00.408838 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:00.400453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38890 (local address 127.24.136.60:38229)
0809 22:52:00.400678 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:00.400684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:00.401766 (+ 1082us) server_negotiation.cc:407] Connection header received
0809 22:52:00.401978 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:00.401985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:00.402073 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:00.402155 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:00.403061 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.404105 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.406129 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:00.406762 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:00.406876 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:00.407498 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:00.407618 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:00.407705 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:00.408476 (+ 771us) server_negotiation.cc:299] Negotiation successful
0809 22:52:00.408563 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:52:00.416798 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31810
I20260809 22:52:00.418057 31910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:00.530439 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31810
I20260809 22:52:00.565218 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 31931
I20260809 22:52:00.566540 32023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:00.676776 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 31931
I20260809 22:52:00.702029 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 32044
I20260809 22:52:00.703333 32152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:00.811174 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 32044
2026-08-09T22:52:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5465 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:52:00 dist-test-slave-pww0 krb5kdc[32515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:52:00 dist-test-slave-pww0 krb5kdc[32515](info): set up 2 sockets
Aug 09 22:52:00 dist-test-slave-pww0 krb5kdc[32515](info): commencing operation
krb5kdc: starting...
W20260809 22:52:02.105641 25120 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/test-user.keytab.
Aug 09 22:52:02 dist-test-slave-pww0 krb5kdc[32515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:52:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.136.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
Entry for principal kudu/127.24.136.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab.
I20260809 22:52:02.462230 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:33925
--webserver_interface=127.24.136.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:33925,127.24.136.61:32893,127.24.136.60:38405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:52:02.830858 32534 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:52:02.831228 32534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:02.831348 32534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:02.831480 32534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:02.838953 32534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:02.839061 32534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:02.839131 32534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:02.839207 32534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:02.839296 32534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:02.849905 32534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:33925,127.24.136.61:32893,127.24.136.60:38405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.62.keytab
--principal=kudu/127.24.136.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.62:33925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.136.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:02.851321 32534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:02.852751 32534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:02.864820 32540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:02.864894 32543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:02.864873 32545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:02.873369 32534 server_base.cc:1034] running on GCE node
I20260809 22:52:02.874301 32534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:02.875955 32534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:02.877197 32534 hybrid_clock.cc:648] HybridClock initialized: now 1786315922877114 us; error 56 us; skew 500 ppm
I20260809 22:52:02.877562 32534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:02 dist-test-slave-pww0 krb5kdc[32515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315922, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:02.880888 32534 init.cc:377] Logged in from keytab as kudu/127.24.136.62@KRBTEST.COM (short username kudu)
I20260809 22:52:02.882570 32534 webserver.cc:460] Webserver started at http://127.24.136.62:40923/ using document root <none> and password file <none>
I20260809 22:52:02.883312 32534 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:02.883417 32534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:02.883766 32534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:02.886889 32534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/data/instance:
uuid: "6e110d3803ab4de98d4e10b3468bed21"
format_stamp: "Formatted at 2026-08-09 22:52:02 on dist-test-slave-pww0"
server_key: "b27dc0a3182690ead4762b12925e5196"
server_key_iv: "c19ae3f249f6f329ca19337707018cc8"
server_key_version: "encryptionkey@0"
I20260809 22:52:02.887781 32534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/wal/instance:
uuid: "6e110d3803ab4de98d4e10b3468bed21"
format_stamp: "Formatted at 2026-08-09 22:52:02 on dist-test-slave-pww0"
server_key: "b27dc0a3182690ead4762b12925e5196"
server_key_iv: "c19ae3f249f6f329ca19337707018cc8"
server_key_version: "encryptionkey@0"
I20260809 22:52:02.893239 32534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:52:02.897238 32559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:02.898797 32534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:52:02.898993 32534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/wal
uuid: "6e110d3803ab4de98d4e10b3468bed21"
format_stamp: "Formatted at 2026-08-09 22:52:02 on dist-test-slave-pww0"
server_key: "b27dc0a3182690ead4762b12925e5196"
server_key_iv: "c19ae3f249f6f329ca19337707018cc8"
server_key_version: "encryptionkey@0"
I20260809 22:52:02.899219 32534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:02.935142 32534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:02.939670 32534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:02.939950 32534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:02.959005 32534 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.62:33925
I20260809 22:52:02.959026 32673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.62:33925 every 8 connection(s)
I20260809 22:52:02.960523 32534 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/data/info.pb
I20260809 22:52:02.964643 32674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:02.968873 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 32534
I20260809 22:52:02.969087 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-0/wal/instance
I20260809 22:52:02.969638 25120 external_mini_cluster.cc:1428] Setting key 9857ea89320cbac0fe5c0138b8747bbc
I20260809 22:52:02.973531 32674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:52:02 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315922, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:52:03.000423 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:02.977876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33925 (local address 127.0.0.1:38066)
0809 22:52:02.978905 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:02.978965 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:02.981828 (+ 2863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:02.983001 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:02.983009 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:02.983908 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:02.984529 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:02.984562 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:02.986462 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:02.986467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:02.987724 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:02.987740 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:02.989044 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:02.989050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:02.989180 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:02.989848 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:02.989873 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:02.992170 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:02.996255 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:02.996273 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:02.996286 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:02.996716 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:02.997204 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:02.997208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:02.997210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:02.997393 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:02.998365 (+ 972us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:02.998371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:02.998587 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 22:52:02.998803 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 22:52:02.999233 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":403,"threads_started":1}
I20260809 22:52:03.000464 32678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:02.977876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38066 (local address 127.24.136.62:33925)
0809 22:52:02.980158 (+ 2282us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:02.980164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:02.980217 (+ 53us) server_negotiation.cc:407] Connection header received
0809 22:52:02.982359 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:02.982366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:02.982713 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:02.982901 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:02.984823 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:02.986271 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:02.987888 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:02.988787 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:02.988974 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:02.992466 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:02.992478 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:02.992495 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:02.992533 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:02.996096 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:02.996879 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:02.996888 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:02.996891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:02.997095 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:02.997527 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:02.997532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:02.997536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:02.998242 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:02.998448 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:02.999001 (+ 553us) server_negotiation.cc:299] Negotiation successful
0809 22:52:02.999394 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":2056,"spinlock_wait_cycles":43136,"thread_start_us":1736,"threads_started":1}
I20260809 22:52:03.003903 32674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:52:03.004993 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.004567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46230)
0809 22:52:03.004738 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.004863 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 22:52:03.005285 32570 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.24.136.61@KRBTEST.COM; defaulting to no policy
I20260809 22:52:03.006986 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.006576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46238)
0809 22:52:03.006735 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.006821 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 22:52:03.007220 32674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32893: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab.
Entry for principal kudu/127.24.136.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab.
I20260809 22:52:03.047140 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.61:32893
--webserver_interface=127.24.136.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab
--principal=kudu/127.24.136.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:33925,127.24.136.61:32893,127.24.136.60:38405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:52:03.056593 32674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } attempt: 1
I20260809 22:52:03.057871 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.057410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46250)
0809 22:52:03.057600 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.057720 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 22:52:03.059032 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.058556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46262)
0809 22:52:03.058743 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.058841 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 22:52:03.059319 32674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32893: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
I20260809 22:52:03.127591 32674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } attempt: 2
I20260809 22:52:03.129128 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.128500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46278)
0809 22:52:03.128817 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.128936 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 22:52:03.130384 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.129923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46292)
0809 22:52:03.130105 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.130214 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 22:52:03.130686 32674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32893: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
I20260809 22:52:03.222007 32674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } attempt: 3
I20260809 22:52:03.223313 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.222841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46294)
0809 22:52:03.223062 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.223167 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 22:52:03.224537 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.224102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46298)
0809 22:52:03.224283 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.224368 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 22:52:03.224745 32674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32893: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
I20260809 22:52:03.368042 32674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } attempt: 4
I20260809 22:52:03.369501 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.368840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46314)
0809 22:52:03.369241 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.369345 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
I20260809 22:52:03.370627 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.370228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46316)
0809 22:52:03.370397 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.370488 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 22:52:03.370817 32674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.61:32893: Network error: Client connection negotiation failed: client connection to 127.24.136.61:32893: connect: Connection refused (error 111)
W20260809 22:52:03.414047 32685 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:52:03.414434 32685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:03.414530 32685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:03.414647 32685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:03.422233 32685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:03.422348 32685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:03.422431 32685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:03.422499 32685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:03.422590 32685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:03.433063 32685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:33925,127.24.136.61:32893,127.24.136.60:38405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.61.keytab
--principal=kudu/127.24.136.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.61:32893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.136.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:03.434494 32685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:03.435945 32685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:03.448019 32693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:03.448050 32696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:03.448333 32692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:03.449050 32685 server_base.cc:1034] running on GCE node
I20260809 22:52:03.449903 32685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:03.451479 32685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:03.452760 32685 hybrid_clock.cc:648] HybridClock initialized: now 1786315923452673 us; error 50 us; skew 500 ppm
I20260809 22:52:03.453150 32685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:03 dist-test-slave-pww0 krb5kdc[32515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315923, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:03.456601 32685 init.cc:377] Logged in from keytab as kudu/127.24.136.61@KRBTEST.COM (short username kudu)
I20260809 22:52:03.458114 32685 webserver.cc:460] Webserver started at http://127.24.136.61:45295/ using document root <none> and password file <none>
I20260809 22:52:03.458837 32685 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:03.458932 32685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:03.459221 32685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:03.462432 32685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/data/instance:
uuid: "8daefc305ebe4e82bef474c2b75d841c"
format_stamp: "Formatted at 2026-08-09 22:52:03 on dist-test-slave-pww0"
server_key: "479594426b71b3de945acdb342e83bb9"
server_key_iv: "91ea49c7a02424ee62772ed412f419b0"
server_key_version: "encryptionkey@0"
I20260809 22:52:03.463214 32685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/wal/instance:
uuid: "8daefc305ebe4e82bef474c2b75d841c"
format_stamp: "Formatted at 2026-08-09 22:52:03 on dist-test-slave-pww0"
server_key: "479594426b71b3de945acdb342e83bb9"
server_key_iv: "91ea49c7a02424ee62772ed412f419b0"
server_key_version: "encryptionkey@0"
I20260809 22:52:03.468743 32685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:52:03.472425 32705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:03.473893 32685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:52:03.474068 32685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/wal
uuid: "8daefc305ebe4e82bef474c2b75d841c"
format_stamp: "Formatted at 2026-08-09 22:52:03 on dist-test-slave-pww0"
server_key: "479594426b71b3de945acdb342e83bb9"
server_key_iv: "91ea49c7a02424ee62772ed412f419b0"
server_key_version: "encryptionkey@0"
I20260809 22:52:03.474287 32685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:03.492707 32685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:03.497421 32685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:03.497695 32685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:03.516137 32685 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.61:32893
I20260809 22:52:03.516170 338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.61:32893 every 8 connection(s)
I20260809 22:52:03.517552 32685 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/data/info.pb
I20260809 22:52:03.521458 339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:03.523231 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 32685
I20260809 22:52:03.523432 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-1/wal/instance
I20260809 22:52:03.523990 25120 external_mini_cluster.cc:1428] Setting key 6dbfbe68415b99f4be70e79968c21193
I20260809 22:52:03.530205 339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:52:03 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315923, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:52:03.553129 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.533524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38068 (local address 127.24.136.62:33925)
0809 22:52:03.534072 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:03.534078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:03.535098 (+ 1020us) server_negotiation.cc:407] Connection header received
0809 22:52:03.537076 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:03.537082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:03.537167 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:03.537289 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:03.538949 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.539845 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.541459 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.541898 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.542021 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:03.545922 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:03.545930 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:03.545935 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:03.545997 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:03.549419 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:03.550315 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:03.550321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:03.550324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:03.550488 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:03.551017 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:03.551022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:03.551025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:03.551489 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:03.551676 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:03.552449 (+ 773us) server_negotiation.cc:299] Negotiation successful
0809 22:52:03.552703 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":272,"threads_started":1}
I20260809 22:52:03.554060 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.534078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33925 (local address 127.0.0.1:38068)
0809 22:52:03.534908 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.534963 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:03.536788 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:03.537497 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:03.537505 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:03.538091 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:03.538697 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:03.538716 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.540023 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.540027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:03.541289 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:03.541303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.542162 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.542171 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:03.542406 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:03.543146 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:03.543170 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:03.545629 (+ 2459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:03.549624 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:03.549633 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:03.549649 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:03.550152 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:03.550669 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:03.550672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:03.550676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:03.550891 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:03.551665 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:03.551672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:03.552064 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 22:52:03.552325 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 22:52:03.552779 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":341,"threads_started":1}
I20260809 22:52:03.556206 339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.136.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.136.60@KRBTEST.COM" created.
Aug 09 22:52:03 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315923, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:52:03.572085 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.556907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46320)
0809 22:52:03.557102 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.557121 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:03.557242 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:03.558488 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:03.558491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:03.558874 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:03.559022 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:03.559031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.560694 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.560698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:03.561621 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:03.561634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.562746 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.562756 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:03.562915 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:03.563538 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:03.563559 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:03.566126 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:03.569405 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:03.569410 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:03.569413 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:03.569869 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:03.570295 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:03.570300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:03.570303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:03.570482 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:03.571104 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:03.571108 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:03.571257 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 22:52:03.571405 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 22:52:03.571718 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 22:52:03.572160 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.557165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46320 (local address 127.24.136.61:32893)
0809 22:52:03.557932 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:03.557941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:03.557975 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:52:03.558042 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:03.558050 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:03.558309 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:03.558483 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:03.559208 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.560548 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.561797 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.562577 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.562689 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:03.566362 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:03.566376 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:03.566392 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:03.566425 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:03.569251 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:03.570018 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:03.570030 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:03.570032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:03.570172 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:03.570606 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:03.570609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:03.570611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:03.570987 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:03.571164 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:03.571622 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 22:52:03.571829 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":418,"threads_started":1}
I20260809 22:52:03.574712 339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:52:03.576061 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.575505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:52912)
0809 22:52:03.575712 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.575861 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 22:52:03.576378 32711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
I20260809 22:52:03.578199 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.577726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:52914)
0809 22:52:03.577916 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.578022 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:52:03.578461 339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38405: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.136.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab.
Entry for principal kudu/127.24.136.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab.
I20260809 22:52:03.600812 25120 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
/tmp/dist-test-tasknxBkGy/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.60:38405
--webserver_interface=127.24.136.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab
--principal=kudu/127.24.136.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.136.20:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.136.62:33925,127.24.136.61:32893,127.24.136.60:38405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:52:03.627872 339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } attempt: 1
I20260809 22:52:03.629218 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.628642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:52924)
0809 22:52:03.628881 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.629012 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 22:52:03.630411 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.629899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:52940)
0809 22:52:03.630077 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.630227 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:52:03.630636 339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38405: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
I20260809 22:52:03.670107 32674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } attempt: 5
Aug 09 22:52:03 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315922, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:52:03.684724 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.671053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.24.136.61:32893)
0809 22:52:03.671277 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:03.671283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:03.671306 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:52:03.671440 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:03.671445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:03.671591 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:03.671686 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:03.672529 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.673261 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.674413 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.674936 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.675050 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:03.678786 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:03.678793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:03.678796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:03.678866 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:03.681628 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:03.682598 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:03.682603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:03.682605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:03.682749 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:03.683191 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:03.683195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:03.683198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:03.683602 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:03.683722 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:03.684252 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 22:52:03.684441 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 22:52:03.684783 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.670937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46334)
0809 22:52:03.671145 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.671170 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:03.671309 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:03.671751 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:03.671755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:03.672181 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:03.672374 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:03.672384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.673399 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.673404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:03.674261 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:03.674276 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:03.675100 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:03.675106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:03.675322 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:03.675958 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:03.675976 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:03.678549 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:03.681834 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:03.681840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:03.681844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:03.682425 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:03.682867 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:03.682872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:03.682876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:03.683059 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:03.683721 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:03.683726 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:03.683932 (+ 206us) client_negotiation.cc:769] Sending connection context
0809 22:52:03.684131 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 22:52:03.684441 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 22:52:03.685932 32674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:52:03.687214 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.686704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:52956)
0809 22:52:03.686905 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.687009 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 22:52:03.688289 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.687901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:52958)
0809 22:52:03.688050 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.688164 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:52:03.688551 32674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38405: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
I20260809 22:52:03.698879 339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } attempt: 2
I20260809 22:52:03.700207 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.699663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:52974)
0809 22:52:03.699879 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.699995 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 22:52:03.701408 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.700981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:52986)
0809 22:52:03.701147 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.701242 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:52:03.701689 339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38405: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
I20260809 22:52:03.739789 32674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } attempt: 1
I20260809 22:52:03.741138 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.740550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53000)
0809 22:52:03.740784 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.740905 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 22:52:03.742285 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.741885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53012)
0809 22:52:03.742040 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.742144 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:52:03.742547 32674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38405: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
I20260809 22:52:03.792954 339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } attempt: 3
I20260809 22:52:03.794267 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.793649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53026)
0809 22:52:03.793923 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.794051 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 22:52:03.795600 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.795014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53040)
0809 22:52:03.795286 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.795395 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
W20260809 22:52:03.795853 339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38405: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
I20260809 22:52:03.810804 32674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } attempt: 2
I20260809 22:52:03.812235 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.811637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53056)
0809 22:52:03.811892 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.812011 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 22:52:03.813323 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.812934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53072)
0809 22:52:03.813096 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.813188 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 22:52:03.813535 32674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38405: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
I20260809 22:52:03.919759 32674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } attempt: 3
I20260809 22:52:03.921119 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.920531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53074)
0809 22:52:03.920773 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.920898 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 22:52:03.922345 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.921858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53082)
0809 22:52:03.922086 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.922191 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 22:52:03.922564 32674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38405: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
I20260809 22:52:03.939121 339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } attempt: 4
I20260809 22:52:03.940457 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.939869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53084)
0809 22:52:03.940141 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.940255 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 22:52:03.941489 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:03.941115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53090)
0809 22:52:03.941268 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:03.941357 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:52:03.941699 339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.136.60:38405: Network error: Client connection negotiation failed: client connection to 127.24.136.60:38405: connect: Connection refused (error 111)
W20260809 22:52:03.966812 353 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:52:03.967166 353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:52:03.967267 353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:52:03.967381 353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:52:03.975061 353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:52:03.975178 353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:52:03.975250 353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:52:03.975322 353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:52:03.975405 353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:52:03.985847 353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.136.20:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.136.62:33925,127.24.136.61:32893,127.24.136.60:38405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/krb5kdc/kudu_127.24.136.60.keytab
--principal=kudu/127.24.136.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.136.60:38405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.136.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:52:03.987252 353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:52:03.988672 353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:52:04.001101 367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:04.001145 370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:52:04.001104 360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:52:04.002445 353 server_base.cc:1034] running on GCE node
I20260809 22:52:04.003409 353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:52:04.005142 353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:52:04.006420 353 hybrid_clock.cc:648] HybridClock initialized: now 1786315924006323 us; error 61 us; skew 500 ppm
I20260809 22:52:04.006831 353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:52:04 dist-test-slave-pww0 krb5kdc[32515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315924, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:52:04.010205 353 init.cc:377] Logged in from keytab as kudu/127.24.136.60@KRBTEST.COM (short username kudu)
I20260809 22:52:04.011955 353 webserver.cc:460] Webserver started at http://127.24.136.60:37225/ using document root <none> and password file <none>
I20260809 22:52:04.012689 353 fs_manager.cc:359] Metadata directory not provided
I20260809 22:52:04.012784 353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:52:04.013074 353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:52:04.016274 353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/data/instance:
uuid: "bac46fc557aa444bbb306267185ad1d7"
format_stamp: "Formatted at 2026-08-09 22:52:04 on dist-test-slave-pww0"
server_key: "c0a61429502a84b27f8cf1206c828c97"
server_key_iv: "67bdf179a244de4f92d15e1c87126d92"
server_key_version: "encryptionkey@0"
I20260809 22:52:04.017099 353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/wal/instance:
uuid: "bac46fc557aa444bbb306267185ad1d7"
format_stamp: "Formatted at 2026-08-09 22:52:04 on dist-test-slave-pww0"
server_key: "c0a61429502a84b27f8cf1206c828c97"
server_key_iv: "67bdf179a244de4f92d15e1c87126d92"
server_key_version: "encryptionkey@0"
I20260809 22:52:04.022657 353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:52:04.026456 381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:52:04.027958 353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:52:04.028138 353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/data,/tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/wal
uuid: "bac46fc557aa444bbb306267185ad1d7"
format_stamp: "Formatted at 2026-08-09 22:52:04 on dist-test-slave-pww0"
server_key: "c0a61429502a84b27f8cf1206c828c97"
server_key_iv: "67bdf179a244de4f92d15e1c87126d92"
server_key_version: "encryptionkey@0"
I20260809 22:52:04.028366 353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:52:04.042677 353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:52:04.046898 353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:52:04.047161 353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:52:04.065415 353 rpc_server.cc:307] RPC server started. Bound to: 127.24.136.60:38405
I20260809 22:52:04.065440 469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.136.60:38405 every 8 connection(s)
I20260809 22:52:04.066852 353 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/data/info.pb
I20260809 22:52:04.070823 470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:52:04.075048 25120 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu as pid 353
I20260809 22:52:04.075259 25120 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknxBkGy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315533505708-25120-0/minicluster-data/master-2/wal/instance
I20260809 22:52:04.075790 25120 external_mini_cluster.cc:1428] Setting key ea8c3e037a00ae9855a6db0a46a8a6bd
I20260809 22:52:04.078402 470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:52:04 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
Aug 09 22:52:04 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315924, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:52:04.098982 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.082592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38078 (local address 127.24.136.62:33925)
0809 22:52:04.084082 (+ 1490us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.084089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.084112 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:52:04.084182 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.084188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.084272 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.084407 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.085720 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.086450 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.088090 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.088668 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.088777 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.093044 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.093050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.093054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.093102 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.095879 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.096611 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.096615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.096617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.096753 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.097146 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.097151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.097154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.097449 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.097563 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.098280 (+ 717us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.098507 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1360,"thread_start_us":1210,"threads_started":1}
I20260809 22:52:04.099762 32674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } attempt: 4
I20260809 22:52:04.100157 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.077766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.24.136.60:38405)
0809 22:52:04.078626 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.078641 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.078692 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:52:04.080643 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.080672 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.081152 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.081554 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.083386 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.085262 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.086721 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.087498 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.087661 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.092639 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.092654 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.092675 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.092726 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.095797 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.096588 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.096597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.096599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.096753 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.097149 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.097153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.097155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.097792 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.098090 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.098367 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.098676 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":595,"thread_start_us":361,"threads_started":1}
I20260809 22:52:04.100183 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.082536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.62:33925 (local address 127.0.0.1:38078)
0809 22:52:04.083343 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:04.083390 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:04.083585 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:04.084479 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:04.084485 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:04.085153 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:04.085519 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.085533 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.086617 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.086623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.087908 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.087924 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.088914 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.088921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.089130 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.089776 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:04.089814 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:04.092813 (+ 2999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:04.096021 (+ 3208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.096032 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.096043 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.096470 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.096871 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.096874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.096876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.097024 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.097573 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:04.097581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:04.097808 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 22:52:04.098141 (+ 333us) client_negotiation.cc:240] Negotiation successful
0809 22:52:04.099004 (+ 863us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":180,"thread_start_us":338,"threads_started":1}
I20260809 22:52:04.102064 470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:52:04 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315922, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.62@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
Aug 09 22:52:04 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315924, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
Aug 09 22:52:04 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.62@KRBTEST.COM
I20260809 22:52:04.116338 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.100319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53108)
0809 22:52:04.100619 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:04.100640 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:04.100743 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:04.102077 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:04.102082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:04.102508 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:04.102728 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.102742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.104084 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.104089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.105338 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.105355 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.106293 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.106301 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.106535 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.107227 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:04.107251 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:04.110025 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:04.113586 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.113592 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.113596 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.113995 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.114366 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.114372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.114376 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.114524 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.115239 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:04.115244 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:04.115494 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 22:52:04.115635 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:52:04.115899 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260809 22:52:04.117359 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.100388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53108 (local address 127.24.136.60:38405)
0809 22:52:04.101790 (+ 1402us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.101796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.101815 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:52:04.101866 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.101872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.101935 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.102023 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.102890 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.103929 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.105491 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.106126 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.106261 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.110235 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.110241 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.110244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.110297 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.113444 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.114115 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.114121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.114125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.114251 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.114663 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.114668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.114671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.115066 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.116641 (+ 1575us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.116762 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.117004 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":1294}
I20260809 22:52:04.117600 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.103084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.24.136.61:32893)
0809 22:52:04.103268 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.103274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.103717 (+ 443us) server_negotiation.cc:407] Connection header received
0809 22:52:04.103934 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.103940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.104048 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.104116 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.105126 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.105938 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.107371 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.107863 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.107975 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.111393 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.111399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.111402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.111446 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.114622 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.115428 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.115434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.115437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.115666 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.116091 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.116096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.116098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.116369 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.116515 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.117091 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.117285 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:52:04.118413 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.102774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.61:32893 (local address 127.0.0.1:46346)
0809 22:52:04.103615 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:04.103634 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:04.103809 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:04.104336 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:04.104340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:04.104770 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:04.104954 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.104967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.106080 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.106085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.107230 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.107243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.108003 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.108008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.108196 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.108715 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:04.108737 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:04.111189 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:04.114738 (+ 3549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.114743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.114746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.115279 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.115831 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.115837 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.115841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.115982 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.116594 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:04.116599 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:04.116787 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 22:52:04.117795 (+ 1008us) client_negotiation.cc:240] Negotiation successful
0809 22:52:04.118085 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":19}
I20260809 22:52:04.119441 470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:52:04.122272 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.103780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.24.136.62:33925)
0809 22:52:04.103992 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.103997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.104250 (+ 253us) server_negotiation.cc:407] Connection header received
0809 22:52:04.104326 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.104330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.104419 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.104513 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.105977 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.106913 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.108719 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.109461 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.109582 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.115908 (+ 6326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.115914 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.115918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.115953 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.119055 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.119866 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.119872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.119876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.120038 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.120549 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.120553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.120556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.120985 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.121248 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.121740 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.121961 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:52:04.123953 32674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21: Bootstrap starting.
I20260809 22:52:04.127777 32674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21: Neither blocks nor log segments found. Creating new log.
Aug 09 22:52:04 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315924, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.60@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
I20260809 22:52:04.128984 32674 log.cc:824] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:04.132223 32674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21: No bootstrap required, opened a new log
Aug 09 22:52:04 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.136.61@KRBTEST.COM
I20260809 22:52:04.136777 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.120655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.24.136.60:38405)
0809 22:52:04.120801 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.120806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.120823 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:52:04.120866 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.120871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.120938 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.121037 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.121878 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.122882 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.124494 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.125096 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.125220 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.129988 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.129994 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.129999 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.130034 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.133484 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.134446 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.134451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.134454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.134601 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.134965 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.134970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.134974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.135314 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.135514 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.136094 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.136439 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":52}
I20260809 22:52:04.137166 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.120406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53124)
0809 22:52:04.120590 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:04.120609 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:04.120709 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:04.121068 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:04.121073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:04.121546 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:04.121723 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.121734 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.123034 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.123038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.124336 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.124351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.125256 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.125264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.125539 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.127108 (+ 1569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:04.127133 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:04.129767 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:04.133814 (+ 4047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.133820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.133824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.134315 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.134729 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.134732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.134735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.134851 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.135707 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:04.135710 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:04.135840 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 22:52:04.135977 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 22:52:04.136288 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 22:52:04.141184 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.124141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46362 (local address 127.24.136.61:32893)
0809 22:52:04.124369 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.124374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.124388 (+ 14us) server_negotiation.cc:407] Connection header received
0809 22:52:04.124464 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.124469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.124544 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.124636 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.126156 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.126880 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.128597 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.129239 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.129387 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.134736 (+ 5349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.134743 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.134747 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.134803 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.138273 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.139054 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.139060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.139063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.139233 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.139721 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.139727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.139731 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.140021 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.140317 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.140620 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.140856 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260809 22:52:04.142367 32674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } }
I20260809 22:52:04.142802 32674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e110d3803ab4de98d4e10b3468bed21, State: Initialized, Role: FOLLOWER
I20260809 22:52:04.143425 32674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } }
I20260809 22:52:04.144508 470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7: Bootstrap starting.
I20260809 22:52:04.144573 485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.145028 485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:52:04.146034 32674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:04.146499 25120 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:52:04.149014 470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:04.149346 32617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6e110d3803ab4de98d4e10b3468bed21"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38084
I20260809 22:52:04.149744 32617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 22:52:04.150182 32617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:04.150803 470 log.cc:824] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:04.153550 32617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } }
I20260809 22:52:04.154476 470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7: No bootstrap required, opened a new log
I20260809 22:52:04.156061 32617 leader_election.cc:288] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [CANDIDATE]: Term 1 election: Requested vote from peers 8daefc305ebe4e82bef474c2b75d841c (127.24.136.61:32893), bac46fc557aa444bbb306267185ad1d7 (127.24.136.60:38405)
I20260809 22:52:04.165726 470 raft_consensus.cc:348] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } }
I20260809 22:52:04.166239 470 raft_consensus.cc:729] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bac46fc557aa444bbb306267185ad1d7, State: Initialized, Role: FOLLOWER
I20260809 22:52:04.166817 470 consensus_queue.cc:260] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } }
I20260809 22:52:04.168025 512 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.168408 512 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:52:04.170858 470 sys_catalog.cc:564] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:04.170907 432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e110d3803ab4de98d4e10b3468bed21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bac46fc557aa444bbb306267185ad1d7"
I20260809 22:52:04.171427 432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:04.175436 432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e110d3803ab4de98d4e10b3468bed21 in term 1.
I20260809 22:52:04.178251 32567 leader_election.cc:302] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e110d3803ab4de98d4e10b3468bed21, bac46fc557aa444bbb306267185ad1d7; no voters:
I20260809 22:52:04.178823 485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:52:04.162277 32534 master_runner.cc:186] Error getting master registration for 127.24.136.61:32893: OK, instance_id { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" instance_seqno: 1786315923510743 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260809 22:52:04.184568 505 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:52:04.184839 505 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:52:04.185694 32534 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:52:04.190850 485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [term 1 LEADER]: Becoming Leader. State: Replica: 6e110d3803ab4de98d4e10b3468bed21, State: Running, Role: LEADER
I20260809 22:52:04.191713 485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } }
W20260809 22:52:04.192976 32574 tablet.cc:2369] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:52:04.202752 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.177611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.24.136.60:38405)
0809 22:52:04.177868 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.177875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.179103 (+ 1228us) server_negotiation.cc:407] Connection header received
0809 22:52:04.179432 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.179439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.179616 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.179688 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.182312 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.183556 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.185709 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.186457 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.186563 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.194390 (+ 7827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.194400 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.194406 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.194458 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.198460 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.199950 (+ 1490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.199958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.199962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.200136 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.200857 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.200863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.200867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.201297 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.201552 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.202118 (+ 566us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.202407 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260809 22:52:04.213876 523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e110d3803ab4de98d4e10b3468bed21. Latest consensus state: current_term: 1 leader_uuid: "6e110d3803ab4de98d4e10b3468bed21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.214475 523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:04.218556 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.179933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38092 (local address 127.24.136.62:33925)
0809 22:52:04.180128 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.180134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.191704 (+ 11570us) server_negotiation.cc:407] Connection header received
0809 22:52:04.191985 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.191992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.192088 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.192162 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.200831 (+ 8669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.202006 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.206006 (+ 4000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.206722 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.206830 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.210936 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.210943 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.210948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.210992 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.214817 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.215828 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.215835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.215839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.216047 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.216528 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.216534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.216537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.216980 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.217273 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.217886 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.218161 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:52:04.223946 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.191880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.24.136.61:32893)
0809 22:52:04.192065 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.192071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.200935 (+ 8864us) server_negotiation.cc:407] Connection header received
0809 22:52:04.201928 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.201936 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.202034 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.202109 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.205580 (+ 3471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.206715 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.209492 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.210234 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.210390 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.216429 (+ 6039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.216438 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.216444 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.216502 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.220454 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.221672 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.221679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.221683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.221871 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.222248 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.222253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.222256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.222687 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.222957 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.223271 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.223565 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:52:04.227015 533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:52:04.229751 533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:52:04.240931 339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } attempt: 5
I20260809 22:52:04.247012 432 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [term 1 FOLLOWER]: Refusing update from remote peer 6e110d3803ab4de98d4e10b3468bed21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:04.248204 523 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 09 22:52:04 dist-test-slave-pww0 krb5kdc[32515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315923, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.136.61@KRBTEST.COM for kudu/127.24.136.60@KRBTEST.COM
I20260809 22:52:04.269544 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.241760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53138 (local address 127.24.136.60:38405)
0809 22:52:04.241955 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.241962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.242204 (+ 242us) server_negotiation.cc:407] Connection header received
0809 22:52:04.242270 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.242275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.242354 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.242452 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.243673 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.244672 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.246382 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.247028 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.247163 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.256351 (+ 9188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:52:04.256360 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:52:04.256366 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:52:04.256413 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:52:04.260707 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.262322 (+ 1615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.262329 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.262334 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.262512 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:52:04.263322 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:52:04.263327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:52:04.263330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:52:04.263780 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:52:04.263971 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:04.268790 (+ 4819us) server_negotiation.cc:299] Negotiation successful
0809 22:52:04.269121 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:52:04.271046 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:04.241645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.136.60:38405 (local address 127.0.0.1:53138)
0809 22:52:04.241952 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 22:52:04.241970 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:52:04.242098 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:52:04.242654 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:52:04.242659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:52:04.243243 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:52:04.243491 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.243506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.244878 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.244884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.246186 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:52:04.246205 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:52:04.247399 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:52:04.247405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:52:04.247657 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:04.248471 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:52:04.248497 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:52:04.252548 (+ 4051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:52:04.261067 (+ 8519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.261073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.261077 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.262059 (+ 982us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.262767 (+ 708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:52:04.262772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:52:04.262776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:52:04.263112 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:52:04.267922 (+ 4810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:52:04.267927 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:52:04.268284 (+ 357us) client_negotiation.cc:769] Sending connection context
0809 22:52:04.269900 (+ 1616us) client_negotiation.cc:240] Negotiation successful
0809 22:52:04.270472 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260809 22:52:04.273406 512 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e110d3803ab4de98d4e10b3468bed21. Latest consensus state: current_term: 1 leader_uuid: "6e110d3803ab4de98d4e10b3468bed21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.273809 512 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:52:04.276660 540 mvcc.cc:205] Tried to move back new op lower bound from 7316750025704636416 to 7316750025500864512. Current Snapshot: MvccSnapshot[applied={T|T < 7316750025704636416}]
I20260809 22:52:04.281546 339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c: Bootstrap starting.
I20260809 22:52:04.282322 548 mvcc.cc:205] Tried to move back new op lower bound from 7316750025704636416 to 7316750025500864512. Current Snapshot: MvccSnapshot[applied={T|T < 7316750025704636416}]
I20260809 22:52:04.288076 339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c: Neither blocks nor log segments found. Creating new log.
I20260809 22:52:04.289592 339 log.cc:824] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:52:04.302228 512 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e110d3803ab4de98d4e10b3468bed21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.302547 512 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:52:04.302695 339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c: No bootstrap required, opened a new log
I20260809 22:52:04.306238 533 catalog_manager.cc:1347] Generated new cluster ID: 26e1c52c2ffc4165a91574d6a090b804
I20260809 22:52:04.306370 533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:52:04.314504 339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } }
I20260809 22:52:04.315142 339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8daefc305ebe4e82bef474c2b75d841c, State: Initialized, Role: FOLLOWER
I20260809 22:52:04.315840 339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } }
I20260809 22:52:04.317565 485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e110d3803ab4de98d4e10b3468bed21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.317896 485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [sys.catalog]: This master's current role is: LEADER
I20260809 22:52:04.330561 533 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:52:04.332679 533 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:52:04.333158 339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:52:04.334844 32760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e110d3803ab4de98d4e10b3468bed21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8daefc305ebe4e82bef474c2b75d841c"
I20260809 22:52:04.335409 32759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:52:04.339231 32759 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [term 1 FOLLOWER]: Refusing update from remote peer 6e110d3803ab4de98d4e10b3468bed21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:52:04.342442 555 catalog_manager.cc:1259] Loaded cluster ID: 26e1c52c2ffc4165a91574d6a090b804
I20260809 22:52:04.343106 32760 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e110d3803ab4de98d4e10b3468bed21 in term 1.
I20260809 22:52:04.349138 555 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7: loading cluster ID for follower catalog manager: success
I20260809 22:52:04.348994 567 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:52:04.350337 566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.350864 566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:52:04.354678 555 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7: acquiring CA information for follower catalog manager: success
I20260809 22:52:04.356343 533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21: Generated new TSK 0
I20260809 22:52:04.324891 353 master_runner.cc:186] Error getting master registration for 127.24.136.61:32893: OK, instance_id { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" instance_seqno: 1786315923510743 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:52:04.357681 353 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:52:04.358963 533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:52:04.361987 555 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bac46fc557aa444bbb306267185ad1d7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:52:04.424702 566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e110d3803ab4de98d4e10b3468bed21. Latest consensus state: current_term: 1 leader_uuid: "6e110d3803ab4de98d4e10b3468bed21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.424969 566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:52:04.424052 576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e110d3803ab4de98d4e10b3468bed21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.425309 576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:52:04.466279 603 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 22:52:04.493691 476 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.487890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.24.136.62:33925)
0809 22:52:04.488185 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.488191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.491560 (+ 3369us) server_negotiation.cc:407] Connection header received
0809 22:52:04.491943 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.491950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.492043 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.492126 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.493403 (+ 1277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38108: BlockingRecv error: recv got EOF from 127.0.0.1:38108 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260809 22:52:04.494259 474 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.485509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.24.136.60:38405)
0809 22:52:04.485712 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.485721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.491273 (+ 5552us) server_negotiation.cc:407] Connection header received
0809 22:52:04.492679 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.492686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.492781 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.492857 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.493988 (+ 1131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53144: BlockingRecv error: recv got EOF from 127.0.0.1:53144 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 22:52:04.496554 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.491870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46374 (local address 127.24.136.61:32893)
0809 22:52:04.492068 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.492075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.494479 (+ 2404us) server_negotiation.cc:407] Connection header received
0809 22:52:04.494874 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.494881 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.494988 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.495061 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.496309 (+ 1248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46374: BlockingRecv error: recv got EOF from 127.0.0.1:46374 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 22:52:04.505528 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.502065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46380 (local address 127.24.136.61:32893)
0809 22:52:04.502263 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.502271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.503947 (+ 1676us) server_negotiation.cc:407] Connection header received
0809 22:52:04.504185 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.504190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.504296 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.504397 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.505268 (+ 871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46380: BlockingRecv error: recv got EOF from 127.0.0.1:46380 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 22:52:04.513309 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.510601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.24.136.61:32893)
0809 22:52:04.510786 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.510793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.511990 (+ 1197us) server_negotiation.cc:407] Connection header received
0809 22:52:04.512199 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.512205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.512278 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.512363 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.513087 (+ 724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46388: BlockingRecv error: recv got EOF from 127.0.0.1:46388 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 22:52:04.557003 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.548632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.24.136.61:32893)
0809 22:52:04.548832 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.548839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.555333 (+ 6494us) server_negotiation.cc:407] Connection header received
0809 22:52:04.555637 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.555644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.555741 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.555807 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.556783 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46404: BlockingRecv error: recv got EOF from 127.0.0.1:46404 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 22:52:04.567793 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.565438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46406 (local address 127.24.136.61:32893)
0809 22:52:04.565704 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.565712 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.566299 (+ 587us) server_negotiation.cc:407] Connection header received
0809 22:52:04.566509 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.566514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.566604 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.566698 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.567551 (+ 853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46406: BlockingRecv error: recv got EOF from 127.0.0.1:46406 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
W20260809 22:52:04.581305 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.579113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.24.136.61:32893)
0809 22:52:04.579336 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.579343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.579912 (+ 569us) server_negotiation.cc:407] Connection header received
0809 22:52:04.580122 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.580127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.580203 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.580294 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.581061 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46412: BlockingRecv error: recv got EOF from 127.0.0.1:46412 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 22:52:04.602739 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.600481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.24.136.61:32893)
0809 22:52:04.600702 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.600710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.601256 (+ 546us) server_negotiation.cc:407] Connection header received
0809 22:52:04.601472 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.601476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.601554 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.601642 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.602469 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46420: BlockingRecv error: recv got EOF from 127.0.0.1:46420 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 22:52:04.647936 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.645584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46424 (local address 127.24.136.61:32893)
0809 22:52:04.645814 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.645821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.646370 (+ 549us) server_negotiation.cc:407] Connection header received
0809 22:52:04.646579 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.646585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.646681 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.646773 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.647659 (+ 886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46424: BlockingRecv error: recv got EOF from 127.0.0.1:46424 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 22:52:04.717460 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.715208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.24.136.61:32893)
0809 22:52:04.715445 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.715476 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.716060 (+ 584us) server_negotiation.cc:407] Connection header received
0809 22:52:04.716271 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.716276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.716357 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.716444 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.717237 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46436: BlockingRecv error: recv got EOF from 127.0.0.1:46436 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 22:52:04.851298 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:04.849076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46444 (local address 127.24.136.61:32893)
0809 22:52:04.849290 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:04.849297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:04.849659 (+ 362us) server_negotiation.cc:407] Connection header received
0809 22:52:04.849869 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:04.849875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:04.849971 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:04.850087 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:04.850888 (+ 801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46444: BlockingRecv error: recv got EOF from 127.0.0.1:46444 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:52:04.957567 567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e110d3803ab4de98d4e10b3468bed21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e110d3803ab4de98d4e10b3468bed21" member_type: VOTER last_known_addr { host: "127.24.136.62" port: 33925 } } peers { permanent_uuid: "8daefc305ebe4e82bef474c2b75d841c" member_type: VOTER last_known_addr { host: "127.24.136.61" port: 32893 } } peers { permanent_uuid: "bac46fc557aa444bbb306267185ad1d7" member_type: VOTER last_known_addr { host: "127.24.136.60" port: 38405 } } }
I20260809 22:52:04.957902 567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e110d3803ab4de98d4e10b3468bed21 [sys.catalog]: This master's current role is: LEADER
W20260809 22:52:05.113000 346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:52:05.111249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46446 (local address 127.24.136.61:32893)
0809 22:52:05.111488 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:05.111497 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:05.111673 (+ 176us) server_negotiation.cc:407] Connection header received
0809 22:52:05.111866 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:05.111871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:05.111954 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:05.112085 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:52:05.112811 (+ 726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46446: BlockingRecv error: recv got EOF from 127.0.0.1:46446 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 22:52:05.471345 603 catalog_manager.cc:1259] Loaded cluster ID: 26e1c52c2ffc4165a91574d6a090b804
I20260809 22:52:05.471498 603 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c: loading cluster ID for follower catalog manager: success
I20260809 22:52:05.474208 603 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c: acquiring CA information for follower catalog manager: success
I20260809 22:52:05.476187 603 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8daefc305ebe4e82bef474c2b75d841c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:52:05.640810 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:05.633446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46448 (local address 127.24.136.61:32893)
0809 22:52:05.634349 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:05.634354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:05.634374 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:52:05.634441 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:05.634448 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:05.634627 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:05.634745 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:05.635289 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.636236 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.638229 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.638715 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.638808 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:05.639513 (+ 705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:05.640150 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:05.640280 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:05.640420 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 22:52:05.640504 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":641,"threads_started":1}
I20260809 22:52:05.654230 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:05.647306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53158 (local address 127.24.136.60:38405)
0809 22:52:05.648002 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:05.648006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:05.648024 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:52:05.648134 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:05.648139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:05.648319 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:05.648496 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:05.648990 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.649796 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.651783 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.652209 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.652322 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:05.652989 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:05.653565 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:05.653697 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:05.653827 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 22:52:05.653911 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":502,"threads_started":1}
I20260809 22:52:05.676558 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:05.661453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53166 (local address 127.24.136.60:38405)
0809 22:52:05.664714 (+ 3261us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:05.664722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:05.669178 (+ 4456us) server_negotiation.cc:407] Connection header received
0809 22:52:05.669298 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:05.669305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:05.669429 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:05.669525 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:05.670407 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.671722 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.674212 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.674803 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.674920 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:05.675688 (+ 768us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:05.675831 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:05.675896 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:05.676146 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 22:52:05.676257 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3049}
I20260809 22:52:05.683449 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:05.661123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.24.136.62:33925)
0809 22:52:05.674131 (+ 13008us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:05.674138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:05.674159 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:52:05.674284 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:05.674290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:05.674515 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:05.675644 (+ 1129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:05.675853 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.677017 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.680685 (+ 3668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.681343 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.681480 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:05.682238 (+ 758us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:05.682764 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:05.682866 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:05.683023 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 22:52:05.683119 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12847,"thread_start_us":13596,"threads_started":1}
I20260809 22:52:05.684100 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:52:05.668218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46460 (local address 127.24.136.61:32893)
0809 22:52:05.668487 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 22:52:05.668495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:52:05.677538 (+ 9043us) server_negotiation.cc:407] Connection header received
0809 22:52:05.677837 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:52:05.677843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:52:05.677946 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:52:05.678018 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:52:05.678741 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.679982 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.681904 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:52:05.682467 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:52:05.682616 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:52:05.683171 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:52:05.683320 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:52:05.683395 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:52:05.683674 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 22:52:05.683783 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 22:52:05.693496 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 32534
I20260809 22:52:05.694638 32667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:05.807485 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 32534
W20260809 22:52:05.842784 32709 connection.cc:535] server connection from 127.0.0.1:46334 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 22:52:05.842793 382 connection.cc:535] server connection from 127.0.0.1:53108 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 22:52:05.843828 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 32685
I20260809 22:52:05.845147 332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:05.957304 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 32685
I20260809 22:52:05.984524 25120 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu pid 353
I20260809 22:52:05.985816 458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:52:06.095161 25120 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknxBkGy/build/asan/bin/kudu with pid 353
2026-08-09T22:52:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5280 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10745 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (392558 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST